-- Logs begin at Mon 2022-07-25 15:27:11 UTC, end at Mon 2022-07-25 16:47:07 UTC. -- Jul 25 16:06:40.544418 dv-cinder-836973-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 16:06:40.558278 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 16:06:40 2022] *** Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: nodename: dv-cinder-836973-15-iscsi Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: machine: x86_64 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: clock source: unix Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: pcre jit disabled Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: detected number of CPU cores: 4 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: current working directory: / Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: detected binary path: /usr/bin/uwsgi-core Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: your processes number limit is 47680 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: your memory page size is 4096 bytes Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: detected max file descriptor number: 2048 Jul 25 16:06:40.561472 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: lock engine: pthread robust mutexes Jul 25 16:06:40.562803 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: thunder lock: enabled Jul 25 16:06:40.562803 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 16:06:40.562803 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 16:06:40.626871 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: Python main interpreter initialized at 0x55a1199479c0 Jul 25 16:06:40.626871 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: python threads support enabled Jul 25 16:06:40.626871 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: your server socket listen backlog is limited to 100 connections Jul 25 16:06:40.626871 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: your mercy for graceful operations on workers is 90 seconds Jul 25 16:06:40.627405 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 16:06:40.627590 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: *** Operational MODE: preforking *** Jul 25 16:06:40.628140 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: *** uWSGI is running in multiple interpreter mode *** Jul 25 16:06:40.628140 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: spawned uWSGI master process (pid: 80212) Jul 25 16:06:40.628390 dv-cinder-836973-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 16:06:40.631678 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: spawned uWSGI worker 1 (pid: 80225, cores: 1) Jul 25 16:06:40.631678 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: spawned uWSGI worker 2 (pid: 80226, cores: 1) Jul 25 16:06:40.631678 dv-cinder-836973-15-iscsi devstack@keystone.service[80212]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 16:06:46.773706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 16:06:46.773706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: from cryptography.utils import int_from_bytes Jul 25 16:06:46.774556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 16:06:46.774556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: from cryptography.utils import int_from_bytes Jul 25 16:06:46.824373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 16:06:46.824596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 16:06:46.824726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] command line args: [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 16:06:46.825018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 16:06:46.825177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ================================================================================ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] admin_token = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_dir = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_source = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] debug = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.826456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_config_append = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_dir = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.828744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.829012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.829261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.829452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.829644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.829878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.830135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.830359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.830593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.830838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.831069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.831320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.831561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.831839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.832360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.832542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] publish_errors = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.832759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.832987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.833171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.833450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.833693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.833930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] standard_threads = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.834162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.834403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.834578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.834759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_journal = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.834934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_json = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.835108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_stderr = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.835270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_syslog = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.835502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:46.835765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.836003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.836279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.836536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.836780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.837020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.837270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.837587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.837878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.838133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.838421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.838657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.838903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.839143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.839397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.839651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.839885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.840144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.840397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.840646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.840888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.841144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.841436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.841691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.841937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.842169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.842401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.842631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.842868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.843119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.843367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.843597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.843837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.844090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.844326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.844566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.844810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.845041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.845285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.845553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.845785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.846017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.846247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.846479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.846719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.846969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.847204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.847442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.847673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.847906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.848141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.848390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.848627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.848878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.849098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.849368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.849691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.849931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.850160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.850412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.850653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.850892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.851123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.851389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.851633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.851859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.852102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.852336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.852608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.852852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.853096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.853475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.853721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.854154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.854395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.854723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.855039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.855282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.855529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.855763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.856012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.856247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.856481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.856720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.856996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.857240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.857511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.857765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.858014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.858257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.858493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.858729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.858964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.859196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.859428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.859680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.859907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.860157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.860395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.860650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.860893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.861147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.861443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.861687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.861924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.external = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.862156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.862458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.862689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.862924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.password = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.863157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.token = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.863398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.863636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.863885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.864125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.864373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.864620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.864850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.865093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.865354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.865611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.865860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.866099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.866345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.866580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.866848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.867101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.867342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.867600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.867887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.868140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.868405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.868648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.868944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.869184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.869449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.869702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.869948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.870176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.870424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.870664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.870918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.871157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.871411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.871648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.871897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.872134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.872386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.872635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.872869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.873285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.873437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.873599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.873836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.874091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.874343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.874574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.874806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.875042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.875282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.875528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.875775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.876046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.876291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.876564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.876802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.877040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.877272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.877555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.877789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.878023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.878272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.878509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.878758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.878987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.879236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.879477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.879709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.879954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.880199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.880429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.880665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.880897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.881133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.881392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.881642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.881880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.882118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.882679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.882928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.883159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.883420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.883640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.883887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.884131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.884364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.884671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.884897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.885142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.885433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.885667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.886079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.886200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.886397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.887493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.887493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.887493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.887493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.887493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.888195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.888195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.888410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.888590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.888838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.889085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.889317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.889596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.889836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.890077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.890310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.890552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.890791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.891044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.891283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.891537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.891793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.892005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.892243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.892481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.892718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.892968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.893205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.893473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.893713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.893954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.894217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.894474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.894711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.894965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.895219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.895456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.895702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.895939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.896179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.896415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.896664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.896896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.897147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.897425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.897678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.897927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.898174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.898421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.898669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.898945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.899136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.899414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.899721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.899881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.900147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.901887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.902042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.902326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.902550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.902813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.903009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.903263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.903451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.903728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.903956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.904238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.904427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.904696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.904920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.905161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.905371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.905661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.905856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.906143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.906393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.906603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.906866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.907095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.907337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.907570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.907807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.908045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.908289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.908534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.908764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.909004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.909305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.909591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.909839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.910081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.910314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.910612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.910769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.911005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.911241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.911474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.911718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.911952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.912194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.912427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.912676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.912910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.913153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.913405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.913647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.913882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:46.914057 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 16:06:46.944212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.944486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.944649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.953730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.984760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.985084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:46.995195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 16:06:46.995195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: from cryptography.utils import int_from_bytes Jul 25 16:06:46.996107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 16:06:46.996107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: from cryptography.utils import int_from_bytes Jul 25 16:06:47.047977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.048872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 16:06:47.049053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 16:06:47.049247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] command line args: [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 16:06:47.049594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 16:06:47.049783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ================================================================================ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 16:06:47.049990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] admin_token = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.050247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_dir = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.050455 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.050636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_source = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.050820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] debug = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.051037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.051278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.051454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.051720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.052034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.052239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.052557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_config_append = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.052845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.053163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_dir = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.053520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.053913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.054261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.054532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.054787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.054969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.055146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.055398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.055648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.055884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.056122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.056401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.056638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.056884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.057156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.057484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.058025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.058258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] publish_errors = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.058508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.058828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.059022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.059378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.059702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.059968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] standard_threads = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.060204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.060508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.060701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.060900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_journal = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.061088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_json = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.061272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_stderr = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.061489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_syslog = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.061689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 16:06:47.061870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.062097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.062367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.062608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.062861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.063102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.063351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.063594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.063843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.064092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.064345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.064579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.064825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.065061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.065308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.065590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.065825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.066097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.066349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.066603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.066840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.067092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.067367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.067623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.067870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.068100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.068331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.068560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.068793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.069040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.069287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.069526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.069766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.070096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.070241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.070479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.070729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.070962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.071200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.071436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.071667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.071956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.072222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.072458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.072700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.072958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.073179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.073452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.073692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.073929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.074170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.074417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.074649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.074886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.075123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.075360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.075596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.075834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.076063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.076320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.076568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.076810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.077042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.077321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.077588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.077881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.078126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.078359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.078624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.078864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.079123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.079374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.079642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.079834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.080073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.080324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.080564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.080852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.081141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.081405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.081649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.081890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.082125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.082392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.082668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.082913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.083157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.083396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.083645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.083880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.084113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.084352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.084669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.084906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.085137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.085445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.085630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.085877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.086115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.086403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.086642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.086902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.087187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.087429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.087707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.external = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.087940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.088243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.088478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.088711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.password = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.088944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.token = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.089185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.089456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.089709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.089941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.090190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.090473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.090708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.091046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.091306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.091559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.091810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.092049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.092295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.092530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.092797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.093049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.093307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.093568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.093867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.094117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.094437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.094683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.094996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.095235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.095477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.095723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.095975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.096206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.096456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.096695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.096950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.097187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.097471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.097711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.097961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.098198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.098456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.098704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.098942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.099180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.099428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.099663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.099903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.100148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.100404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.100636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.100869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.101807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.101807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.101807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.102180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.102369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.102516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.102754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.102889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.103018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.103144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.103315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.103572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.103977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.104247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.104487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.104860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.105025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.105209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.105472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.105721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.105971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.106206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.106483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.106731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.106966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.107206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.107438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.107676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.107913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.108254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.108497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.108735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.108990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.109240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.109730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.109979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.110227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.110492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.110883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.111165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.111647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.111647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.111980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.112280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.112535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.112785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.113036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.113291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.113725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.114031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.114285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.114602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.114839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.115174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.115587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.115842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.116095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.116345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.116581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.116832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.117109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.117396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.117698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.117809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.118052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.118328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.118591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.118882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.119112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.119359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.119596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.119933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.120128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.120450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.121097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.121097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.122747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.123001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.123160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.123650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.124226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.124226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.124689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.124818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.125115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.125618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.126054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.126413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.126787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.126787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.127049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.127303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.127555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.127880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.128138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.128437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.128691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.128942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.129405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.129649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.129901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.130155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.130445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.130701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.130940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.131258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.131609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.131855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.132215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.132761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.132997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.133248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.133549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.133826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.134069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.134308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.134598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.134836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.135075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.135358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.135599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.135858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.136107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.136347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.136592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.136938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.137189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.137457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.137703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.137941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.138190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.138440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.138671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.138918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.139218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.139479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.139733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.139987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.140212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.140456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.140694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.140931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.141166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.141427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.141673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.141910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.142150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.142384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.142633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.142866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.143109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.143339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.143587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.143820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 16:06:47.144052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 16:06:47.172689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.173055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.173266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.183318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.185838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.185992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.186199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.186500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.186725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.186940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.207396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.207629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.214574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.214804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.215035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.215262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.215484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.215697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.215906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.228065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.241630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.241848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.242077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.247343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.247343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.247343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.247343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.247901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.261847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.262115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 16:06:47.262529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.262806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 16:06:47.263145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.263473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.263962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.264289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.276071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.276365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.276650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.276928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.286857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.289467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.289767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.290054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.290344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.290625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.290919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.291186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.291459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.291730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.292004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.292272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.293513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.293831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.321908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.323564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.323842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.329468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.329898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.330185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.337656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.337950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.341804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.342035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.342274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.342501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.342637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.342727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.342918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.343002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.343210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.343494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.343776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.344047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.344318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.344596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.356817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.357043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.357271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.357522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.357737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.357950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.358168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.363854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.364174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.368264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.369159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.372184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.372446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.372806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.373146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.373289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.373475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.373646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.373822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.374201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 16:06:47.379112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.383228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.386991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.387306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.387582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.387866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.388137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.388458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.389232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.389520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 16:06:47.390100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.390450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 16:06:47.391086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.391672 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.392098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.392397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.405364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.405662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.405945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.406231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.407993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.408292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.408570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.408834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.414916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.416925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.418341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.418625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.418712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.418909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.418994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.419190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.419521 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.419750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.420029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.420326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.420597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.420876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.421143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.422176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.422456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.422750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.423037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.423303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.437036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.437484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.437760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.437997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.438304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.438605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.438894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.452906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.454608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.454889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.457601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.460629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.460923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.461047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.461312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.464435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.466257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.467950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.468259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.468510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.469237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.469586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.475235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.475641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.475911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.476162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.476399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.476597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.476846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.477129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.477465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.478001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.481518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.481856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.482226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.482503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.491921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.492285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.492523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.500865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.501289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.501474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.501830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.502117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.502635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.503202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.503691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.504191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.504408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.504643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.504842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.505142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.505644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.506146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.506611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.510121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.514249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.514666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.520317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.524313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.528337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.528699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.529029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.529399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.529750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.530074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.551939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.552317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.552529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.552757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.561321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.563199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.563473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.567065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.567500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.567788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.568071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.568346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.582328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55a1199479c0 pid: 80226 (default app) Jul 25 16:06:47.583616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.583931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.584351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.584634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.585023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.585313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.585632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.587913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82dcb-1907-45aa-9622-e4581a4c5681 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:47.588432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82dcb-1907-45aa-9622-e4581a4c5681 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:47.588923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82dcb-1907-45aa-9622-e4581a4c5681 None None] PATH_INFO: `/v3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:47.596779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Mon Jul 25 16:06:47 2022] GET /identity/v3/ => generated 255 bytes in 14 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:06:47.604897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.609210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.612690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.614716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.616246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.616527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.616810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.624372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.626114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.629700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.630012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.630522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.630801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.641402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.641796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.642088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.651131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 16:06:47.651477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.651701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.651959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.652229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.652633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.652907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.653196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.653499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.653770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.654064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.654343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.654627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.654889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 16:06:47.696164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55a1199479c0 pid: 80225 (default app) Jul 25 16:06:55.698735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:55.699281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:55.699624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:55.701635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 2/2] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:06:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:06:55.710459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:55.711117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:55.711117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:55.785531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: INFO dbcounter [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] Registered counter for database keystone Jul 25 16:06:55.811188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG oslo_db.sqlalchemy.engines [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80225) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 16:06:55.821648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writer thread running {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 16:06:56.103365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG passlib.handlers.bcrypt [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80225) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 16:06:56.103905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG passlib.handlers.bcrypt [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80225) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 16:06:56.174591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:06:56.177519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:06:56.380612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1/3] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:06:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 674 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 16:06:56.537683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:06:56.611835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: INFO dbcounter [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] Registered counter for database keystone Jul 25 16:06:56.633493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG oslo_db.sqlalchemy.engines [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80226) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 16:06:56.641008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writer thread running {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 16:06:56.763406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37c417-6a88-4ab6-882a-d704f81656b3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:06:56.768120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:06:56.768818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:56.769107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:56.769411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:56.781187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:06:56.830976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 16:06:56.831327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b6b5b-391b-4d87-b67c-f4dfe39ac223 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:06:56.842110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 3/4] 10.222.0.66 () {58 vars in 1324 bytes} [Mon Jul 25 16:06:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 16:06:56.848315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:06:56.863750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cfec5-7e44-45a1-beb6-0af34af425db None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:06:56.869099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:06:56.870484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:56.870776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:56.871057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] PATH_INFO: `/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:56.872720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] RBAC: Authorizing `identity:get_project(project_id=b8ee5c3834cf43d4a8d8dd7aee49ae24)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:06:56.925861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING keystone.common.rbac_enforcer.enforcer [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 16:06:56.926117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a1c9c-fc88-4429-8b9f-f70ee1ec8546 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:06:56.931247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 2/5] 10.222.0.66 () {56 vars in 1207 bytes} [Mon Jul 25 16:06:56 2022] GET /identity/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:06:59.008413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71adfee-1425-4321-8dc3-3e82facc7514 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:59.008996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71adfee-1425-4321-8dc3-3e82facc7514 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:59.008996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71adfee-1425-4321-8dc3-3e82facc7514 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:59.013289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 4/6] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:06:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:06:59.019248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:59.019740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:59.019849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:59.042575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:06:59.044198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:06:59.053162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 3/7] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:06:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 25 16:06:59.076106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71adfee-1425-4321-8dc3-3e82facc7514 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:06:59.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71adfee-1425-4321-8dc3-3e82facc7514 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:06:59.096860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:06:59.097566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:59.097863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:59.098132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:59.107527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:06:59.109294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c6ad-4235-4c96-85f6-0d3f309295c3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:06:59.120061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 5/8] 10.222.0.66 () {58 vars in 1324 bytes} [Mon Jul 25 16:06:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:06:59.126682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:06:59.136924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6ea8-4e3f-4228-b5f8-90275885373d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:06:59.141941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:06:59.142852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:06:59.143131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:06:59.143405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:06:59.144989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:06:59.147681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9308f12-5ef8-44de-b2e3-9976ee7f3994 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:06:59.150052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 4/9] 10.222.0.66 () {56 vars in 1198 bytes} [Mon Jul 25 16:06:59 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:01.349421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fe4b7-5726-4607-99eb-20864f5e5c03 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.350026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fe4b7-5726-4607-99eb-20864f5e5c03 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.350026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fe4b7-5726-4607-99eb-20864f5e5c03 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.351358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 6/10] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:01.357456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.357961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.358069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.377326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab62c89-d4e9-46df-96d0-b18b06a53b4f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.377785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab62c89-d4e9-46df-96d0-b18b06a53b4f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.378077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab62c89-d4e9-46df-96d0-b18b06a53b4f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.378681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:01.379835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 7/11] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:01.379987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:01.387080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.387516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.387653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.435627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG passlib.handlers.bcrypt [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80226) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 16:07:01.436043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG passlib.handlers.bcrypt [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80226) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 16:07:01.482578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:01.484197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:01.486763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 5/12] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:01.514393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:01.545605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:01.601940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 8/13] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:01.625783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:01.642510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41682125-5dad-4f4a-96fe-e73f98e9e0d2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:01.686794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:01.687615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.687910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.688187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.701280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:01.722940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:01.724006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:01.724959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:01.725618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 6/14] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:01.732093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:01.748267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3186d9-04c7-4bc0-b7ac-cb9ddf2c4c31 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:01.751103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:01.801886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:01.802912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.803233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.803543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.866322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:01.898840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb051a6-69cc-4169-825d-7c31c208ae83 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:01.912886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:01.914564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:01.944298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:01.945062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:01.945525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:01.945911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:01.947329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:01.947781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:01.949487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:01.966301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 7/15] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:01.972752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:02.040861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 9/16] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 16:07:02.046787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:02.050312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:02.111613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdce88-5c8f-49ff-8e7c-4201c7cd4c7c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:02.171749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:02.173243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:02.173607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:02.174023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:02.175941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:02.178461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:02.181361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 8/17] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:01 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:02.191727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:02.208750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2599d-2d6c-4137-a5f0-79adf49db56f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:02.220836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:02.252135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:02.253485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:02.254138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:02.254297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:02.278073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:02.279667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:07:02.308196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:02.309881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:02.311427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:07:02.311427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:07:02.311427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:07:02.311427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:02.311427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 10/18] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:02.319230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:02.357576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8caa8-222c-4844-9e94-2d6310189385 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:02.399892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:02.417179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:02.418209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:02.418499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:02.418779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:02.420069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:02.421720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:02.423467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 9/19] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:02.430375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:02.483130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2dba-98c7-4c27-a883-feb31b844c70 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:02.534217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:02.543232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:02.544329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:02.544821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:02.545244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:02.546444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:02.547043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:02.549657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58ae4-a249-44a4-9b75-85d1ef3a1a97 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:02.558193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a8c5b-b0d9-48f2-a108-f88f2ef4d384 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:02.579238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 11/20] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:02.601199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:02.602078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:02.602454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:02.602903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:02.604040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.domain.id=default, user.id=3c939fd3ba4449cf949d6e5bd73effec) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:02.604478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:02.606359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d744534-82e3-4970-9bcd-89bb24d244d4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:02.615121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 10/21] 10.222.0.66 () {56 vars in 1322 bytes} [Mon Jul 25 16:07:02 2022] GET /identity/v3/role_assignments?user.id=3c939fd3ba4449cf949d6e5bd73effec&scope.domain.id=default&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 233 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:06.139922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24da-b2f9-4fb4-8809-ab2eeae89e49 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.141054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24da-b2f9-4fb4-8809-ab2eeae89e49 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.141817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24da-b2f9-4fb4-8809-ab2eeae89e49 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.143818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 12/22] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:06.152592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.153761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.154252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.181850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:06.183481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:06.272917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3725bca-9de4-439d-8d78-ba1cbb51cc0a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.273693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3725bca-9de4-439d-8d78-ba1cbb51cc0a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.274162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3725bca-9de4-439d-8d78-ba1cbb51cc0a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.275945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 13/23] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:06.287412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.288487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.289298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.290492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 11/24] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:06.337932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:06.340460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:06.397120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:06.501771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 14/25] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:06.576261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:06.592002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:06.654403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bb7f7-332d-4dcc-b67b-291a06a2a1e2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:06.720923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:06.817309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce710b-1a4f-4d94-9aee-961ef355b482 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:06.874516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:06.874790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:06.875619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.876248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.876407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.876776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.877013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.877409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.878605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:06.881264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:06.898793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-01b997ad-e215-4862-8d84-37ac3cd25e6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:06.908652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:06.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:06.910758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:06.912727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 12/26] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 516 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:06.914401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66cfb-079b-4feb-b0e5-f62020cfbe22 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.914923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66cfb-079b-4feb-b0e5-f62020cfbe22 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.915400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66cfb-079b-4feb-b0e5-f62020cfbe22 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.916977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 13/27] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:06.922717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc472cc-28d3-455d-a34b-81dba3a79cbb None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:06.923317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc472cc-28d3-455d-a34b-81dba3a79cbb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:06.923798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc472cc-28d3-455d-a34b-81dba3a79cbb None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:06.925393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 14/28] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:06.926391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc472cc-28d3-455d-a34b-81dba3a79cbb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:07.051389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc472cc-28d3-455d-a34b-81dba3a79cbb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:07.102553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:07.103750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.104251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.104875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.106006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:07.106541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:07.108252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0492c-83cc-4028-b996-2b379c596413 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:07.110379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:07.121853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 15/29] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:07.124058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869eb7-85c3-4790-8a61-e2a19081c2ab None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.124655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869eb7-85c3-4790-8a61-e2a19081c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.125166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869eb7-85c3-4790-8a61-e2a19081c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.149396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8c869eb7-85c3-4790-8a61-e2a19081c2ab None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:07.151485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8c869eb7-85c3-4790-8a61-e2a19081c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:07.269369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 16/30] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:07.269369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ebd5-402c-4a8c-b2e3-2ff42a80e598 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.269369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ebd5-402c-4a8c-b2e3-2ff42a80e598 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.269369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ebd5-402c-4a8c-b2e3-2ff42a80e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.288081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6098ebd5-402c-4a8c-b2e3-2ff42a80e598 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:07.289711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6098ebd5-402c-4a8c-b2e3-2ff42a80e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:07.325013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 15/31] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 25 16:07:06 2022] POST /identity/v3/roles => generated 219 bytes in 750 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:07:07.326816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:07.405843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 17/32] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:07.408476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbdd0a-be22-4911-b3c5-f47ed4a3eae8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.409500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbdd0a-be22-4911-b3c5-f47ed4a3eae8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.410281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbdd0a-be22-4911-b3c5-f47ed4a3eae8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.420145 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 18/33] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:07.421558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbdd0a-be22-4911-b3c5-f47ed4a3eae8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:07.510365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:07.517290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac3c5-222d-4634-b0e2-d7685c9257c5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:07.533611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:07.570590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbdd0a-be22-4911-b3c5-f47ed4a3eae8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:07.591432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:07.593460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.594220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.594890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.598374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:07.601224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c57e5-6ecf-45eb-b71c-9d9dd42837d0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:07.604730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 16/34] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:07 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:07.607017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.607498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.607986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.620577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:07.621292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.621623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.621895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:07.622834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:07.624406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:07.645267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:07.647948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:07.746316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 17/35] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:07.747232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:07.757842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 19/36] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 25 16:07:07 2022] POST /identity/v3/roles => generated 225 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:07.759154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:07.925719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f05ef1-6747-4e19-9192-aca33493a27b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:07.929898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:07.950994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2007f7-d0eb-4d46-a222-0f355cb38a37 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:07.980374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:07.986593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:07.988066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:07.990664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:07.990664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.009201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.009748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.010836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.011271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.011743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.011892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.013620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.016234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc85c8-ef3f-4917-8206-ab3754cc8d8c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.018495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 20/37] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:08.021001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df95e0-3394-4576-a126-ba8d349752c5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.021526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df95e0-3394-4576-a126-ba8d349752c5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.022019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df95e0-3394-4576-a126-ba8d349752c5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.025579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 21/38] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:08.026753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df95e0-3394-4576-a126-ba8d349752c5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:07:08.046708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:08.048106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:08.048908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:07:08.048908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:07:08.048908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:07:08.048908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:08.048908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 18/39] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:08.049232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.159548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df95e0-3394-4576-a126-ba8d349752c5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.179176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b29cbd-65c2-4f4a-8495-c3e29fe8bf87 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.197479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.198174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.198462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.198741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.199505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.201109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800fca-dfcd-442e-a330-d8cf08f646c5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.216562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.217534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.217828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.218116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] PATH_INFO: `/v3/domains/default/users/3c939fd3ba4449cf949d6e5bd73effec/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.233854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 22/40] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 25 16:07:08 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:08.234063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3c939fd3ba4449cf949d6e5bd73effec, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.234151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.234151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.234151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.234348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.242304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 19/41] 10.222.0.66 () {58 vars in 1361 bytes} [Mon Jul 25 16:07:08 2022] PUT /identity/v3/domains/default/users/3c939fd3ba4449cf949d6e5bd73effec/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:08.243657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.262958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:08.266275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:08.285425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:08.358563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:08.421399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 23/42] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:08.448345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.458338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72946df9-797a-4575-af2c-f12681a0527c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.513454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.514289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.514670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.514948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.515698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:08.516009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.517475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.533260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 20/43] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:08.540916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.593993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed42bb6-c462-4da9-9e09-040db9ba6dff None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.632116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:08.636829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.637721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.638007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.638280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.639615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.641143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.642859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 24/44] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:08.648594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:08.661356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b146c7-8cb1-41cd-a4cb-2007e7b35fc9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.699812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:08.706824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.707499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.707805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.708081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.708805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.710361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5d4de-122e-4f48-b2c9-05be09784dc6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.758772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 21/45] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:07:08 2022] POST /identity/v3/projects => generated 301 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:08.774634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb62c9a-76f1-4184-83c9-90c3caf7ad7f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:08.818154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:08.818847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:08.819147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:08.819418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:08.820153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:08.821825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f484fe-c694-42e3-aa80-5d5ab4609f6e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:08.856132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 25/46] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:07:08 2022] POST /identity/v3/projects => generated 312 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:10.364331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d503bd-13da-42ce-b457-168537b87a10 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:10.364877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d503bd-13da-42ce-b457-168537b87a10 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:10.364877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d503bd-13da-42ce-b457-168537b87a10 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:10.366408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 22/47] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:10.372644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:10.373479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:10.373479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:10.395527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:10.396749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:10.485857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 26/48] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:10.527985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d503bd-13da-42ce-b457-168537b87a10 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:10.729152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d503bd-13da-42ce-b457-168537b87a10 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:10.783030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:10.799254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:10.800717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:10.801215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:10.801737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:10.826484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ac610d5f-8672-4098-b428-5ac2ac9d02cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:10.844463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:10.845830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:10.846668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:10.847274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 23/49] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:10.855350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:10.890069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:10.980096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978519b-2988-407a-9af8-a25fdff52b7a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:10.980328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978519b-2988-407a-9af8-a25fdff52b7a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:10.980602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978519b-2988-407a-9af8-a25fdff52b7a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:10.983463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 24/50] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:10.988298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:10.988589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:10.988872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:10.989304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347e197-c780-4f36-a57a-8a212a5cb1ec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.010274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:11.011487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:11.030475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.031166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.031453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.031770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.033710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:11.033710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.034513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.044286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 27/51] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:11.050098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:11.100810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 25/52] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:11.124785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:11.178360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:11.196021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9332e0b-5cdf-48f3-b523-c6d86573ec02 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.239569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.240919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.241366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.241801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.244090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.244301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff346-fdf7-416d-a0e2-854cb5f4f56d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.246946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.250417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 28/53] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:11 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:11.256131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:11.271431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:11.292500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.293314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.293631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.293931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.295156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.296665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.298427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 26/54] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:11.304436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:11.376427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cb0d-fa1b-4370-a78e-bd2271bbdcb6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.416759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.417646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.417944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.418227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.419503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.421040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.423204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 29/55] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:11.429562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:11.445537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296ae9-5dfa-4275-b889-bce8cf22ae4a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.462610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:11.504912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.505917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.506427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.506915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.508469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.511075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0069e-3729-44b4-a4ba-d0ddaf5d4e06 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.552920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 27/56] 10.222.0.66 () {60 vars in 1140 bytes} [Mon Jul 25 16:07:11 2022] POST /identity/v3/projects => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:07:11.570361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792395c-29f4-4899-9548-ee5a5aa37529 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:11.614885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:11.615684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:11.616078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:11.616473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:11.617587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.domain.id=default, user.id=3c939fd3ba4449cf949d6e5bd73effec) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:11.618010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:11.619723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaedde8-a610-4e0a-9b5f-ffa7ff94b46d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:11.627183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 30/57] 10.222.0.66 () {56 vars in 1322 bytes} [Mon Jul 25 16:07:11 2022] GET /identity/v3/role_assignments?user.id=3c939fd3ba4449cf949d6e5bd73effec&scope.domain.id=default&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 530 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:13.693208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29989a-fa8b-4138-bd5d-d99c9bbf9c10 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:13.693867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29989a-fa8b-4138-bd5d-d99c9bbf9c10 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:13.693867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29989a-fa8b-4138-bd5d-d99c9bbf9c10 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:13.695399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 28/58] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:13.701914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:13.702730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:13.702730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:13.724159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:13.725420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:13.826285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 31/59] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:13.860942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29989a-fa8b-4138-bd5d-d99c9bbf9c10 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:13.912697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:13.999592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29989a-fa8b-4138-bd5d-d99c9bbf9c10 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:14.043874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:14.044700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:14.044992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:14.045275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:14.046564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:14.048092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41aaf39-6a54-495e-9656-e468e7049e0e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:14.049845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 29/60] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:14.056159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:14.077084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:14.181007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994d7b6-2586-4c55-b1c5-d0c550b327f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:14.226115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:14.226816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:14.227103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:14.227387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:14.228149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:14.229743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9705cc-a353-4d3c-a8cd-99d989b6778f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:14.258976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 32/61] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:14 2022] POST /identity/v3/users => generated 275 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:17.825251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4bede-0bd1-498d-91a1-95dca6691cbf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:17.826929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4bede-0bd1-498d-91a1-95dca6691cbf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:17.827758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4bede-0bd1-498d-91a1-95dca6691cbf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:17.831166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 30/62] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:17.840991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db1833-420c-4cf3-aa4a-980b16687357 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:17.842584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db1833-420c-4cf3-aa4a-980b16687357 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:17.843367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db1833-420c-4cf3-aa4a-980b16687357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:17.852508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43be30-38cd-4c9d-a308-53a0b2c7494b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:17.853502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43be30-38cd-4c9d-a308-53a0b2c7494b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:17.854320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43be30-38cd-4c9d-a308-53a0b2c7494b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:17.857287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 31/63] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:17.872583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:17.873720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:17.874481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:17.882029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e0db1833-420c-4cf3-aa4a-980b16687357 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:17.885034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0db1833-420c-4cf3-aa4a-980b16687357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:17.908556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:17.914760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:18.043979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 33/64] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:18.045743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102027-c8a0-4dc0-94cc-cab9aef815a9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.046306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102027-c8a0-4dc0-94cc-cab9aef815a9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.046824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102027-c8a0-4dc0-94cc-cab9aef815a9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.048494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 34/65] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:18.054835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.055373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.055961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.072974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:18.096950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:18.098161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:18.115512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 32/66] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:18.117674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.193321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:18.237859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 35/67] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:18.239500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.323281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39936-fa26-4983-932f-21c533d079ae None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:18.398208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:18.400450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.401387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.402339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.423364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:18.449506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:18.450471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:18.451250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:18.453644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 33/68] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:18.454727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.464117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090286b-e9db-4244-a59f-25ebe12374ec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:18.524175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:18.533197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:18.534993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.535789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.536495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.557987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:18.573067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:18.574844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:18.576256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:18.580972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 36/69] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:18.583130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.656789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:18.665808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b17c2-7188-4ded-b275-79a4b0c84c34 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:18.716123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:18.716996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.717251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.717543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.730695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c8492ee4-14c9-41d1-a9a2-26b15efd99e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:18.739038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:18.739982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:18.740891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 34/70] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:18.741802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc893-c55d-4da6-bebe-28b130776160 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.742031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc893-c55d-4da6-bebe-28b130776160 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.742276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddc893-c55d-4da6-bebe-28b130776160 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.743752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 35/71] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:18.744526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddc893-c55d-4da6-bebe-28b130776160 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.753631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d6a3-a3ef-4003-bb46-aff326b05f1c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:18.803404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:18.804581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.805090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.805708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.807179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:18.807750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:18.810642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc968b-ce2d-46e3-8f29-01a34eb71d38 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:18.827312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 37/72] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:18.829797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086e3a-1b2e-4292-8005-77b9d0f50e6b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.830422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086e3a-1b2e-4292-8005-77b9d0f50e6b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.830777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086e3a-1b2e-4292-8005-77b9d0f50e6b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.833388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 38/73] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:18.834791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086e3a-1b2e-4292-8005-77b9d0f50e6b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:18.887459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddc893-c55d-4da6-bebe-28b130776160 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:18.930749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:18.941709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:18.942464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.942755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.943030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.943940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:18.944255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:18.945760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b760b-d2db-49a9-94fd-cd51f7d591f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:18.960348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 36/74] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:18.962408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cc8-4b56-4bc1-8502-8d133abb4977 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:18.962925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cc8-4b56-4bc1-8502-8d133abb4977 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:18.963412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2955cc8-4b56-4bc1-8502-8d133abb4977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:18.985865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a2955cc8-4b56-4bc1-8502-8d133abb4977 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:18.987300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a2955cc8-4b56-4bc1-8502-8d133abb4977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:18.992448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086e3a-1b2e-4292-8005-77b9d0f50e6b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.003625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.033530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.034344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.034750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.035147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.036062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:19.036497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.038102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.047962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 39/75] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:19.049138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.082239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 37/76] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:19.089174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.089734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.090257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.121911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:19.123368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:19.191868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7056-fe42-49c1-bf00-e6ae0efb7fe0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.216936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.222993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 38/77] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:19.224998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.251894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.253756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.254288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.255001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.258018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.261431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.268419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 40/78] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:19.268419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.355645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.383039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b266eb4-198b-4e97-aed0-c0a9816abeeb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.398622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a45-8ac5-4eb5-bc9e-0cc7968e6858 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.427387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.428317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.428606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.428885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.453858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.454811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.454972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.455085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.455372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.456655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.457894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.458242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.460032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 41/79] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:19.467436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.467627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 39/80] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:19.467627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.551035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.616266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae798-9dcf-4744-ac00-363726688422 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.625996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52782-d578-4ba7-a258-ef26ed535e35 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.668051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.669637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.670156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.670293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.670656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.671038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.671344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.671624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.672859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.674409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.676159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 40/81] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:19.676159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.676159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.692405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:19.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:19.709381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:19.710288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 42/82] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:19.712268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:19.828818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c5ca-af86-4fb6-9e55-00ab67ff0ec9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.859443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2d4df-56b4-497f-8787-1eb1ffb4ba76 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:19.877582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.878435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.879041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.879337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.885515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:19.916111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:19.916801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:19.917089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:19.917456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:19.971450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:19.972750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:19.976088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.038635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.040763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.129243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 41/83] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:20.130536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.193041 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 43/84] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:20.194356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.219746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.267662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207b4b7-e0a3-4f47-aec6-711c5761e5af None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:20.270666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.310972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:20.311907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:20.312326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:20.312704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:20.344323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d137bdb-9a95-4338-bcf6-49e7902454c7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:20.399833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:20.400651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:20.400982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:20.401306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:20.402152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.404091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.430088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.431882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.445547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 44/85] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:20 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:20.445547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.566900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.586468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 42/86] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:20.587402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.601399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.642099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c5892-26b3-44e9-b8d6-9bcdda3169d8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:20.714907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:20.716070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:20.716597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:20.717092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:20.718606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:20.719250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.721884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.741388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 45/87] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:20.742858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.763572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a28-7fd8-4763-8533-b52a6a540465 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:20.830652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:20.831596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:20.831895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:20.832178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:20.833847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.835388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.839098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 43/88] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:20.839098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:20.919981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0456-a493-4a56-ac56-694afcadde26 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:20.961649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.971288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:20.975714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:20.977831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:20.978558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:20.979291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:20.982660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:20.987463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:20.992362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 46/89] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:20.994254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:21.020297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b8f8-ab5c-4bae-8fee-de02de15f2c2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.067468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.068484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.068778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.069073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.070619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.072188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:21.074094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 44/90] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:21.074932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:21.219326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27eaa0-548f-4adb-b39a-697aee9ca236 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.249189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f2-48a3-4787-b9bd-0281d175ffc9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.279880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.280827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.281118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.281440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.282636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=3c939fd3ba4449cf949d6e5bd73effec) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:21.282957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.284631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:21.294680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 45/91] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:21 2022] GET /identity/v3/role_assignments?user.id=3c939fd3ba4449cf949d6e5bd73effec&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:21.295718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:21.312632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:21.335903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.338368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.339195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.339938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.344210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.349641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:21.355772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 47/92] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:20 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:21.358356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:21.485581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:21.536455 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7dccf-e471-4af6-9579-e77920004467 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.587176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.588571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.589790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.590089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.590986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:21.591421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.592997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:21.600821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 48/93] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:21 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:21.601848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:21.620712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e49d2c-34ac-463c-9b0b-560a6597348b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.684081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.686080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.686671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.688890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:21.689486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.692096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:21.701774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 46/94] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:21 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 259 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:21.774354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6eb4-d555-4ee0-a831-28e8b2c386a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:21.825680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:21.833850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:21.835647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:21.836430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:21.837557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:21.936378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:21.938190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:22.093968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 49/95] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:22.099269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:22.279941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:22.361041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d9cd9-8c54-48d8-9917-1aaa92e10cb5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:22.427326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:22.428871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:22.429743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:22.429992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] PATH_INFO: `/v3/projects/326968c85e6c44f7b1e8b127022c6806` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:22.432516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=326968c85e6c44f7b1e8b127022c6806)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:22.435367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47476ee7-dbe1-4993-ae7f-dffb7a046ea1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:22.438552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 47/96] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:22 2022] GET /identity/v3/projects/326968c85e6c44f7b1e8b127022c6806 => generated 312 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:22.452858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:22.525712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:22.640908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107525de-3db7-4eac-961e-1069c13d2ec7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:22.695432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:22.696232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:22.696572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:22.696884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:22.697777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=326968c85e6c44f7b1e8b127022c6806, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:22.698100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:22.699836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421512e-629f-4ee8-be68-8d4096191808 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:22.711042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 50/97] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:22 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=326968c85e6c44f7b1e8b127022c6806&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:24.402702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7697-021e-471f-9c8e-571c14af08ac None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:24.403774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7697-021e-471f-9c8e-571c14af08ac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:24.404462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7697-021e-471f-9c8e-571c14af08ac None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:24.406276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 48/98] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:24.413434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:24.414070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:24.414548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:24.418597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bd31-f1fd-494f-a891-68ccc7ee1241 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:24.420138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bd31-f1fd-494f-a891-68ccc7ee1241 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:24.421250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bd31-f1fd-494f-a891-68ccc7ee1241 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:24.430948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 51/99] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:24 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:24.446784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:24.447524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:24.448028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:24.459100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:24.472240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:24.492220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:24.493959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:24.657054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 49/100] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:24.675752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 52/101] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:24.728241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:24.744913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:24.819450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:24.870280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:24.995837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9bcc4-84bb-4784-a183-13d915237119 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:25.024280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a25e3a-ae0b-4afb-afe5-65c64065cde2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:25.104302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:25.105826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.106346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.106844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.109209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:25.112313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:25.116171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 53/102] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:25.125810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:25.134606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:25.136001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.136448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.136895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.182568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:25.206979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:25.208053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:25.208860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:25.210262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 50/103] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:25.219207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:25.310301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:25.327226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c4d5-3f37-4301-acd2-966c2eba7e42 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:25.439091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2118-2f71-46bb-9bf0-c347c0846edc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:25.457794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:25.458883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.459364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.459836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.461084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:25.464005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca6c20-6327-41bb-8c6b-397845d7f9ff None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:25.494043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:25.522170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:25.523040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.523445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.523871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.524860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:25.525308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:25.526931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8972db2-4b59-4c2e-a9c8-f5a5192a9e95 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:25.543551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 51/104] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:25.546165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d38d-6fbe-443d-9588-9909b0d6d26f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.546863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d38d-6fbe-443d-9588-9909b0d6d26f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.547551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d38d-6fbe-443d-9588-9909b0d6d26f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.550136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 52/105] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:25.550904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 54/106] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:07:25 2022] POST /identity/v3/projects => generated 302 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:07:25.552625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8f73-b84b-48ae-9512-254206871e83 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.553176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8f73-b84b-48ae-9512-254206871e83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.553734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8f73-b84b-48ae-9512-254206871e83 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.554456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edaee-1fcf-406b-91af-cd57b19824d5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.555642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edaee-1fcf-406b-91af-cd57b19824d5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.556456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edaee-1fcf-406b-91af-cd57b19824d5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.556657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 53/107] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:25.558323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b8f73-b84b-48ae-9512-254206871e83 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:25.562395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 55/108] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:25.568044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652fa3-6a6a-4dae-9da8-f0937b80b8f4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.569038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652fa3-6a6a-4dae-9da8-f0937b80b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.569963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652fa3-6a6a-4dae-9da8-f0937b80b8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.602970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-57652fa3-6a6a-4dae-9da8-f0937b80b8f4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:25.605832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-57652fa3-6a6a-4dae-9da8-f0937b80b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:25.742311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 56/109] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:25.745255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.746105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.746814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.774989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:25.776821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:25.778551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b8f73-b84b-48ae-9512-254206871e83 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:25.845633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:25.846484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.846760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.847042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] PATH_INFO: `/v3/users/f41b3966013e4b5bb64b88c0140ce716` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.878107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f41b3966013e4b5bb64b88c0140ce716)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:25.881718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb247ffe-2fb1-4eb0-980c-84a5d11291c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:25.886369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 54/110] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:25 2022] GET /identity/v3/users/f41b3966013e4b5bb64b88c0140ce716 => generated 289 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:25.889940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:25.891101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:25.891747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:25.902866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 57/111] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:25.904566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:25.920815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:25.930848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:25.933544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:26.031824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:26.080675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 55/112] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:26.082432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:26.128454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffa606-f640-4ef2-8fc6-aba5e39ba8e2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:26.192962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:26.194560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.195168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.195737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.215446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:26.230124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:26.232033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:26.233840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:26.236348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 58/113] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:26.243619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:26.296076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8edb83-3c08-4bb6-97d2-693afbb36f3a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:26.359786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:26.364699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:26.366323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.366974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.368036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.443113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a491-b47d-4416-804b-38d10271334a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:26.459627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:26.507023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:26.510592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:26.516093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:26.518431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.518431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.518873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.538079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:26.553780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:26.555750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:26.557861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:26.559737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 59/114] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:26.566568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:26.708041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 56/115] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:26.709266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:26.752331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67865114-3a21-4aa6-8553-da76a881eb7d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:26.799081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:26.818976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:26.820578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.821227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.821768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.831440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:26.843416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:26.853117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:26.855002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:26.856447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 60/116] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:26.857695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:26.885145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145cf32-1475-4e57-b83d-51174ddfa257 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:26.944738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:26.945967 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.946477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.947141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.948636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:26.949195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:26.951671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e086-db44-4676-a592-7ab083e9a8cb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:26.976544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 57/117] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:26.979064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f348c-d92c-427e-a074-dd83a81e59b0 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:26.979581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f348c-d92c-427e-a074-dd83a81e59b0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:26.980085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f348c-d92c-427e-a074-dd83a81e59b0 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:26.982891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 58/118] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:26.985324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f348c-d92c-427e-a074-dd83a81e59b0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.031732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d11cf-2a6f-4de5-ad19-fab4856a97ca None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.080218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.081522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.082026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.082540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.083861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:27.084383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.086914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.101394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 61/119] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.102708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.160490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f348c-d92c-427e-a074-dd83a81e59b0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.178764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:27.220776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.221883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.222306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.222725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.224523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.226413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.228768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 59/120] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:26 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.235633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.247248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:27.263018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599aee-35d9-4adc-9019-648a1830c427 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.312283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.313215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.313669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.314123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.315242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:27.315789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.317797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e1c8-915a-4c75-8a85-8f9271a0550b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.329113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 62/121] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.335897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.336338 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.336494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.357656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:27.359310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:27.395681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e50c-dada-4e9f-96db-ad12cc698001 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.451792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.452752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.453040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.453326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.454763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.456376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.458253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 60/122] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.459045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.476127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 63/123] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:27.478439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.552404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:27.576461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:27.612219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe5ca7-1624-4b94-a93b-bce23f3a7f57 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.639343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315d878-c32b-4f48-ad4c-26acba1b6fd0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.657085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.658130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.658413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.658692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.660286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.661718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.668085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 61/124] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.668085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.684316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.684981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.685281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.685612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.686440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=f41b3966013e4b5bb64b88c0140ce716) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:27.686757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.688247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.694870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 64/125] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/role_assignments?user.id=f41b3966013e4b5bb64b88c0140ce716&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.695561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.850475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986ecd4-71ec-48ec-b61d-82873b9ad9e5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.864217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec3207-bef0-4ba7-b86b-3ce24b5d9b9a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:27.866960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:27.903661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.904567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.905100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.905414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.907091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.908576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.910340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 62/126] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.911110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.917199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:27.918987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:27.919646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:27.920227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:27.922051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:27.923818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:27.925947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 65/127] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:27.926976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:27.938491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:28.059036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369ae9-7132-4e46-9f3e-53a19870d5d0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.062264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b2a6c-31e2-4263-b5da-bb15f38e7122 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.099522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.100839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.101077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.101399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.102725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.104245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ff1b5-cbce-4c7b-813a-54cbc66fd0db None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.105226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.106210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 66/128] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:28.106316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.106386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.106624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.112355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14b546-bd5b-494b-ad97-a365bee90a46 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.112915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14b546-bd5b-494b-ad97-a365bee90a46 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.113255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14b546-bd5b-494b-ad97-a365bee90a46 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.115746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 67/129] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:28.116975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14b546-bd5b-494b-ad97-a365bee90a46 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.122847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:28.130710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:28.131749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 63/130] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:28.133258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.174017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:28.223350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:28.273249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14b546-bd5b-494b-ad97-a365bee90a46 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.285014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4442-cbce-4d40-b612-455ffe07bc96 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.322438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.323682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.324150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.324721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.327798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.338735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 68/131] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:28 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:28.338735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.366115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.367595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.368163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.368568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/dded27d91b9449209b35a9c238089d76/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.371985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user_id=dded27d91b9449209b35a9c238089d76, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.375059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068a3c4-52c8-4144-8840-e365bf71e655 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.405378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 64/132] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:28 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/dded27d91b9449209b35a9c238089d76/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:28.405378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.405378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.405378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.423575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:28.425484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:28.559259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 65/133] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:28.560767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.581470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861a4b-a76e-4063-9264-98135e1c393b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.599303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:28.660240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.662338 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.662840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.663677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/dded27d91b9449209b35a9c238089d76/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.667805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:28.668845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user_id=dded27d91b9449209b35a9c238089d76, role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.672027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.702373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 69/134] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:28 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/dded27d91b9449209b35a9c238089d76/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:28.703871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.738131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618a89-006a-48f9-9b14-08dea6497d6a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.809021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.809776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.810066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.810351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.811560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:28.811677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.813644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.841822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf92b-62b4-408b-a446-d56289b0738c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:28.849052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 66/135] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:28.850717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.890638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:28.892334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:28.892930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:28.893441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/3c939fd3ba4449cf949d6e5bd73effec/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:28.896491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user_id=3c939fd3ba4449cf949d6e5bd73effec, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:28.898538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:28.916450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 70/136] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:28 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/3c939fd3ba4449cf949d6e5bd73effec/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:28.917950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:28.963654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:29.065253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2dc6-86e6-4a88-b725-086f887b7408 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:29.098123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf751c8-6b7d-4d2e-82e1-3afd0397053d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:29.113105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:29.114916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:29.115457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:29.116192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:29.117933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:29.120150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:29.122865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 71/137] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:28 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:29.129159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:29.132674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:29.153216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:29.154131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:29.154424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:29.154710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:29.156416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:29.158081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:29.160357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 67/138] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:29.184455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:29.288333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:29.309315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccdf2d-7c20-4001-8f6e-d9ce9fc2a16c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:29.364398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:29.365750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:29.366043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:29.366324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] PATH_INFO: `/v3/projects/326968c85e6c44f7b1e8b127022c6806` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:29.367726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] RBAC: Authorizing `identity:get_project(project_id=326968c85e6c44f7b1e8b127022c6806)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:29.369564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:29.373178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 72/139] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:29 2022] GET /identity/v3/projects/326968c85e6c44f7b1e8b127022c6806 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:29.385162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:29.500591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64666f-bccd-4525-96bf-99122a05da6f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:29.625948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:29.627141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:29.627616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:29.628103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:29.629077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:29.630980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3f118-749c-4ed1-a45b-aa11eaf708d9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:29.642929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70443cdd-00f8-4116-a783-1368b9e204fd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:29.691715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 68/140] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:29 2022] POST /identity/v3/users => generated 283 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:07:29.735251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:29.736263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:29.736591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:29.736873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] PATH_INFO: `/v3/projects/326968c85e6c44f7b1e8b127022c6806/users/dded27d91b9449209b35a9c238089d76/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:29.743323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] RBAC: Authorizing `identity:create_grant(project_id=326968c85e6c44f7b1e8b127022c6806, user_id=dded27d91b9449209b35a9c238089d76, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:29.745016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c810c1c-ff63-47a7-99a5-db6993e352ed None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:29.766426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 73/141] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:29 2022] PUT /identity/v3/projects/326968c85e6c44f7b1e8b127022c6806/users/dded27d91b9449209b35a9c238089d76/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:30.705430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1a2a-878b-4eb2-9e55-1587475c1026 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:30.706629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1a2a-878b-4eb2-9e55-1587475c1026 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:30.707129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1a2a-878b-4eb2-9e55-1587475c1026 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:30.709253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 69/142] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:30.717867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:30.718848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:30.719326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:30.743369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:30.745166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:30.770768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:30.843683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 74/143] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:30.883605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa1a2a-878b-4eb2-9e55-1587475c1026 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:30.952844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:31.020048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa1a2a-878b-4eb2-9e55-1587475c1026 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:31.066320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:31.067653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:31.068191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:31.068809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:31.083684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:31.094182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:31.095417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:31.096275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:31.098957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 70/144] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:31.106770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:31.255063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e71f3e-368a-47b2-beef-fd91eb17bb0e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:31.307306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:31.308375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:31.308858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:31.309328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:31.310356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:31.310905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:31.312713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:31.327138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 75/145] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:31.333654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:31.567330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084160c-7e40-4c8a-bb38-08f0c3df007b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:31.576635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:31.618182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:31.619122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:31.619408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:31.619844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] PATH_INFO: `/v3/users/f41b3966013e4b5bb64b88c0140ce716` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:31.626469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] RBAC: Authorizing `identity:get_user(user_id=f41b3966013e4b5bb64b88c0140ce716)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:31.628009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:31.630141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 71/146] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:31 2022] GET /identity/v3/users/f41b3966013e4b5bb64b88c0140ce716 => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:31.636222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:31.675601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:31.799524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb696d9-b00a-40e0-8e8e-7017e1af648c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:31.853785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:31.855192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:31.855532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:31.855848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:31.857500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:31.859358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018e30-4b9f-4c48-a593-dd3ac53ad814 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:31.861811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 76/147] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:31 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:31.871799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:32.072337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b394af-5dad-4bf4-9380-6f3e05649a00 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:32.150123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:32.151604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:32.152096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:32.152610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/f41b3966013e4b5bb64b88c0140ce716/roles/01fc9b48f93348c88c5cb563a080e941` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:32.156437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user_id=f41b3966013e4b5bb64b88c0140ce716, role_id=01fc9b48f93348c88c5cb563a080e941)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:32.159878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bc884-e5a1-4b63-8c4a-1ab60856c123 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:32.181800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 72/148] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:31 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/users/f41b3966013e4b5bb64b88c0140ce716/roles/01fc9b48f93348c88c5cb563a080e941 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:32.440941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12625d9c-d0aa-41da-ab08-019ab08d09ab None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:32.441234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12625d9c-d0aa-41da-ab08-019ab08d09ab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:32.441548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12625d9c-d0aa-41da-ab08-019ab08d09ab None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:32.443178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 77/149] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:32.450141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:32.450513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:32.450783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:32.486984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:32.488702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:32.547115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:32.630550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 73/150] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:32.673131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12625d9c-d0aa-41da-ab08-019ab08d09ab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:32.820566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:32.845538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12625d9c-d0aa-41da-ab08-019ab08d09ab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:32.886959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:32.887808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:32.888084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:32.888361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:32.901868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:32.910589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:32.912232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:32.915046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 78/151] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:32.920611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:33.206241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ef41-273f-40fc-baf9-c498d7c481ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:33.292258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:33.293551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.294067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.294732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.296307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:33.296907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:33.304012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:33.321422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 74/152] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:33.327155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:33.530723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1e046-b2f9-4204-a602-b4ce99d4bb7c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:33.574654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:33.575566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.575846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.576178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.577497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:33.579029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef984e2-0442-422a-9f93-b256988f3d61 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:33.580768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 79/153] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:33 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:33.586691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:33.620330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:33.654311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599eee8-7b75-46c9-9868-59605ea88914 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.654608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599eee8-7b75-46c9-9868-59605ea88914 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.654933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599eee8-7b75-46c9-9868-59605ea88914 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.656264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 80/154] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:33.662187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090c3-51cc-457d-87b7-7c0d4afa5f56 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.662579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090c3-51cc-457d-87b7-7c0d4afa5f56 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.662754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090c3-51cc-457d-87b7-7c0d4afa5f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.696861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-02a090c3-51cc-457d-87b7-7c0d4afa5f56 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:33.698137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-02a090c3-51cc-457d-87b7-7c0d4afa5f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:33.711119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:33.771254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a5c8-435a-4ae0-bee5-706877c9d29d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:33.870258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 81/155] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:33.902355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:33.903054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.903348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.903631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.922942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9478f-58c2-409d-805b-9a220302c73e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:33.923482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9478f-58c2-409d-805b-9a220302c73e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:33.924297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9478f-58c2-409d-805b-9a220302c73e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:33.928803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 82/156] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:33.930192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9478f-58c2-409d-805b-9a220302c73e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:34.094663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:34.099169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:34.131429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9478f-58c2-409d-805b-9a220302c73e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:34.179747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:34.207453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:34.208318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:34.208637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:34.208926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:34.229763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:34.238544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c61bbe4c-fda1-4007-a49c-23298d7612ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:34.255734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:34.256730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:34.257900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 83/157] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:34.260612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:34.260612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:34.260612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:34.318596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:34.320742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:34.445134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 75/158] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 859 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:34.446472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:34.463385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 84/159] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:34.464376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:34.664855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cae61-9d62-49a7-bf46-53b5f0fa0a73 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:34.745980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:34.767830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a651-a43f-4646-bd35-8f55949ba974 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:34.778511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:34.779927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:34.780453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:34.781031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:34.782709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:34.784850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:34.787219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 85/160] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:34 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:34.793591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:34.847278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:34.866201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:34.866909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:34.867201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:34.867479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:34.868437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:34.868791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:34.870390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392db1e-7408-4825-9635-1420457d8d75 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:34.902209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 76/161] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:34.905507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bb44-2d00-4245-9c80-06d7ae5a2389 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:34.906373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bb44-2d00-4245-9c80-06d7ae5a2389 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:34.907057 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477bb44-2d00-4245-9c80-06d7ae5a2389 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:34.915868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 77/162] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:34 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:34.918037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477bb44-2d00-4245-9c80-06d7ae5a2389 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:35.127130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6cda-c235-4be6-88fa-a5f01618533d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:35.169175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477bb44-2d00-4245-9c80-06d7ae5a2389 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:35.230159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:35.231014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:35.231307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:35.231569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:35.256059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:35.256790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:35.257869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:35.258200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:35.258525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:35.259469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:35.259788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:35.261497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de16779-51a8-4722-9a11-a4d1834b987a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:35.280330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 78/163] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:34 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 608 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:35.281852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:35.282142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:35.282415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:35.283311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:35.284928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:35.286538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:35.302248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 86/164] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:35.304317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:35.340563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:35.342044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:35.506681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:35.528179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:35.593073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 79/165] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:35.594744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:35.624658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2025-ec69-48d8-8b1e-8e47ee3270d1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:35.704379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:35.706592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:35.707364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:35.708162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:35.710916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:35.714854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f4fba-e8d8-4ff7-bdb2-efd19032683e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:35.722288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 87/166] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:35 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:35.727543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c706-cd14-4a54-bde9-3287e9008534 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:35.728580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c706-cd14-4a54-bde9-3287e9008534 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:35.728835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c706-cd14-4a54-bde9-3287e9008534 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:35.732976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 88/167] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:35.739647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2c706-cd14-4a54-bde9-3287e9008534 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:35.930661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacb3c9-db7e-477e-b03d-7e699e35cb36 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.010370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2c706-cd14-4a54-bde9-3287e9008534 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.024247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.026077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.027063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.028005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.029885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:36.030869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:36.034188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:36.045896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.060473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 80/168] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:36.063111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.071558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.071558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.071558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.072020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.093044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f9b57e17-c9e3-4adc-a4bf-ab85077f0c08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:36.105859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:36.106812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:36.107608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:36.110214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 89/169] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:36.111996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.112462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.117374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.154535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:36.156849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:36.170361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.247005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 90/170] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:36.248665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.291823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45f91-554c-4d44-995c-09fa5c4c5cb8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.355028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.356531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.357256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.357988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.374821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6797d9-e3d8-4eee-9f7d-57e42e7c6464 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.419670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.420880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.421374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.421916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] PATH_INFO: `/v3/users/3c939fd3ba4449cf949d6e5bd73effec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.423432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] RBAC: Authorizing `identity:get_user(user_id=3c939fd3ba4449cf949d6e5bd73effec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:36.425145 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:36.427285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 91/171] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/users/3c939fd3ba4449cf949d6e5bd73effec => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:36.428403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.433597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.503984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:36.506644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:36.573075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e308-c321-4f52-8e6d-0242e9fa76b0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.591598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.620162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.621431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.621795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.622275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.623579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:36.624094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:36.626673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810086-ba47-47b3-8de8-2eb7be1cf4bc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:36.641843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 92/172] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:36.644159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ae3-e687-4084-a0e3-2fd32b7a3782 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.644623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ae3-e687-4084-a0e3-2fd32b7a3782 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.645038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ae3-e687-4084-a0e3-2fd32b7a3782 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.647401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 93/173] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:36.648660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30ae3-e687-4084-a0e3-2fd32b7a3782 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.666909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 81/174] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:36.667814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.728121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.797237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f03d0-cfdb-4840-870b-5bce28e7d3b4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.802443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30ae3-e687-4084-a0e3-2fd32b7a3782 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:36.847006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.847901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.848335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.848700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.849236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:36.850352 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.850744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.851109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.862534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8dc05a5f-24b4-43ce-a091-0166c1b1e7e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:36.870736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:36.872465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:36.874328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 94/175] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:36.876030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13bef-b869-4b1c-929e-7977e950004f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.876323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13bef-b869-4b1c-929e-7977e950004f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.876733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13bef-b869-4b1c-929e-7977e950004f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.878919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 95/176] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:36.880639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef45186-af65-40f7-af24-b73925ce43b9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:36.881091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef45186-af65-40f7-af24-b73925ce43b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:36.881703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:36.882446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef45186-af65-40f7-af24-b73925ce43b9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:36.884702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 96/177] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:36.886299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef45186-af65-40f7-af24-b73925ce43b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:36.960726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:36.962467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c08b92-33c4-4222-bab9-4f7336d77ca3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.041266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef45186-af65-40f7-af24-b73925ce43b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.050922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.083273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.084324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.084684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.085132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.104264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:37.105895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.107513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 82/178] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:37.108334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 97/179] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:36 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:37.109360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.109735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.110260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.110770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.142362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:37.144473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:37.208941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.259915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 83/180] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:37.261263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.267370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a760ba4-d491-4e67-8a2f-0dade77738c3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.311492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.312610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.312944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.313277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.314905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:37.316833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0b5bb-265b-498d-988f-0d37ae913945 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.319086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 98/181] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:37.320699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0a8b5-ea38-4fe8-8ddd-6a626afecf44 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.321173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0a8b5-ea38-4fe8-8ddd-6a626afecf44 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.321627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0a8b5-ea38-4fe8-8ddd-6a626afecf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.341963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4df0a8b5-ea38-4fe8-8ddd-6a626afecf44 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:37.343206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4df0a8b5-ea38-4fe8-8ddd-6a626afecf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:37.379263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.428713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a827599-73d0-4d29-b336-1f0207a41355 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.431150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 99/182] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:37.432592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.432890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.433168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.464532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:37.465890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:37.479033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.479736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.480124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.480404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.481291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:37.481638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:37.483139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.502725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 84/183] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:37.502725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.559070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 100/184] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:37.561225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.570119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.675635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.689015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b019b-4690-4fc2-9b90-2898fd3ce2ca None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.691676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d4f13-1f3a-42cb-863d-ea8fb3de5bf4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.731941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.732966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.733313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.733756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.735236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:37.736850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.739205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 85/185] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:37.740575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.747956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.748589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.749008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.749444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.842768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:37.845692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:37.876269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addcba5-a6df-4d43-b650-517bbbccd32b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:37.880422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:37.929693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:37.930638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:37.931018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:37.931396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:37.944524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:37.964216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:37.965731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:37.967102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 86/186] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:37.968368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:37.990701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:38.004884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 101/187] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:38.005989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.134523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b7854-1898-40ce-87d4-42f93e453ec2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.145415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176dc55-1972-41ac-945a-9888822134ef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.187199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.187964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.188355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.188723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.189897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:38.190388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.192006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.197152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.198063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.198347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.198740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.200173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 87/188] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:37 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.201181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.217620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:38.226354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:38.227426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 102/189] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:38.228714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.252279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:38.307126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:38.370352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf29b5-2949-48a0-9ec1-23c39e38e024 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.372377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab767-1760-4b1d-852d-70a5b3c9c349 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.418397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.419241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.419729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.420098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.422222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.424127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.433165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 103/190] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.433165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.449534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.451395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.451763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.452104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] PATH_INFO: `/v3/users/dded27d91b9449209b35a9c238089d76` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.453998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] RBAC: Authorizing `identity:get_user(user_id=dded27d91b9449209b35a9c238089d76)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.455555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.459158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 88/191] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/users/dded27d91b9449209b35a9c238089d76 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.460088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.604476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a146fa-f492-40b1-b066-c5b96a7c4444 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.622690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:38.632010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472e66b-2ba0-4162-ab02-8e387247a33c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.634237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:38.654519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.655390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.655752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.656132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.657326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=3c939fd3ba4449cf949d6e5bd73effec) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:38.657840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.659608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.668169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 104/192] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/role_assignments?user.id=3c939fd3ba4449cf949d6e5bd73effec&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.669661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.682173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.682902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.683186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.683587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.684595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:38.684873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.686420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.702525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 89/193] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.703471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.863326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d47c22-ae43-419e-88f6-6a88c3bf8877 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.868606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48f9cf-b475-4708-b233-3ec2c271dc83 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:38.922603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.924378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.925505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.925684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.926062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:38.927118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:38.927558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:38.927709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.927994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:38.929289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:38.929820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:38.930423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.933776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 105/194] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.934887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.935357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:38.952996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 90/195] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:38.953848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:38.992204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.021929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.080603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72705e-8b2a-4cfe-bad6-b9613566bc2f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.106492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd04a9-e1fe-495e-9d6a-0ee2f52fec45 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.124137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:39.124869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:39.125155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:39.125469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:39.126215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:39.127731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:39.152896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 106/196] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:38 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:39.154253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:39.160161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:39.161371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:39.161900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:39.162397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:39.295569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:39.297839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:39.319417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaefce5-3d91-4003-b90a-28c797c575c0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.337009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.337273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.381715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:39.383175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:39.383638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:39.384053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:39.385980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:39.388318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:39.395221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 107/197] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:39 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:39.395221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:39.506953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 91/198] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:39.509107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:39.568632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255f50-43b1-486e-ab8c-37b39e66343b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.620854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:39.621594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:39.621980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:39.622307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:39.623175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=20428d815d0d44b28a7d873242e06a9c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:39.623498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:39.625034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:39.632713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 108/199] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:39 2022] GET /identity/v3/role_assignments?user.id=20428d815d0d44b28a7d873242e06a9c&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:39.633614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:39.726419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.732406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c9558-3a0e-42cb-be30-dbcf5a4f0811 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.771403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:39.794261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:39.795365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:39.795831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:39.796183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] PATH_INFO: `/v3/users/f41b3966013e4b5bb64b88c0140ce716` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:39.798029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] RBAC: Authorizing `identity:get_user(user_id=f41b3966013e4b5bb64b88c0140ce716)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:39.800448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:39.802937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 92/200] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:39 2022] GET /identity/v3/users/f41b3966013e4b5bb64b88c0140ce716 => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:39.805078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:39.917704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b232dbb-9a28-4b08-8979-3e590a33105a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:39.965094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d391a8-dedb-45e3-b2f1-93a7cc1214fc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:40.052534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:40.053710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:40.054198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:40.054674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:40.098369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:40.099894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:40.100380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:40.100847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] PATH_INFO: `/v3/projects/326968c85e6c44f7b1e8b127022c6806` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:40.104622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] RBAC: Authorizing `identity:get_project(project_id=326968c85e6c44f7b1e8b127022c6806)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:40.107446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:40.114386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 93/201] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:39 2022] GET /identity/v3/projects/326968c85e6c44f7b1e8b127022c6806 => generated 312 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:40.119331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:40.229871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:40.315280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:40.317938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:40.319483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee213f9-0233-43c3-809e-f77b8523368e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:40.397385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:40.400103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:40.401220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:40.401883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:40.421879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:40.604563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:40.607133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:40.629406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 109/202] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 996 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:40.631331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:40.770108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:40.828247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fc09-1f14-4c51-aa4b-45965fdad36e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:40.899254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 94/203] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:40.900623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:40.922659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:40.923747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:40.924216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:40.924681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:40.926313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=326968c85e6c44f7b1e8b127022c6806, user.id=dded27d91b9449209b35a9c238089d76) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:40.926862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:40.929703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:40.964099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 110/204] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:40 2022] GET /identity/v3/role_assignments?user.id=dded27d91b9449209b35a9c238089d76&scope.project.id=326968c85e6c44f7b1e8b127022c6806&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:40.964099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:40.982811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:41.269558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63537d23-a40a-43ea-8606-f7a7b905367b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:41.288845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ef2e0-ecd9-4238-8c5e-025db8536080 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:41.373797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:41.376297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:41.377050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:41.377733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:41.380595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:41.384142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:41.389056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 111/205] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:40 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:41.393390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:41.393390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:41.394374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:41.394374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:41.396166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:41.401385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:41.401385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:41.403940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 95/206] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:40 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:41.414384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:41.517764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:41.551887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:41.612984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3fec-3211-4de2-a003-bbca84ef4e05 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:41.679150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:41.680319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:41.680801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:41.681254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:41.684144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c, user.id=f41b3966013e4b5bb64b88c0140ce716) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:41.684675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:41.687553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455865d7-9fee-4ded-96c4-95277bf5d5c9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:41.699752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 112/207] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:41 2022] GET /identity/v3/role_assignments?user.id=f41b3966013e4b5bb64b88c0140ce716&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:41.709320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84b86-76fa-4ef3-b240-ee77ab8641b7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:41.780743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:41.782009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:41.782517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:41.783033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:41.784691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=20428d815d0d44b28a7d873242e06a9c) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:41.785214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:41.788056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2088b-d98f-4004-985b-16998cfcbd4f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:41.800691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 96/208] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:41 2022] GET /identity/v3/role_assignments?user.id=20428d815d0d44b28a7d873242e06a9c&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 259 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:42.305980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec039-9adf-463e-a8e2-f35e3813bbbf None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:42.306906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec039-9adf-463e-a8e2-f35e3813bbbf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:42.307564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec039-9adf-463e-a8e2-f35e3813bbbf None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:42.309578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 113/209] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:42.317714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:42.318419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:42.318974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:42.350842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:42.352562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:42.458271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 97/210] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:42.486263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec039-9adf-463e-a8e2-f35e3813bbbf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:42.618395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec039-9adf-463e-a8e2-f35e3813bbbf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:42.659349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:42.660437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:42.660736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:42.661092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:42.664682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:42.675243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:42.683528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:42.685980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:42.687431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 114/211] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:42.691626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:42.727372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:42.824049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b8dde-5c6c-494a-872d-5a2a9a188d06 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:42.864058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:42.864878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:42.865155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:42.865457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:42.866245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:42.866650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:42.868652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:42.878869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 98/212] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:42.885671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.017360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a42909-77ef-46ff-9f14-844540c49eb4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.057789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.058822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.059096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.059360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.060905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:43.062623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3787082-6915-4970-9090-1c074fae8f33 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:43.064830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 115/213] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:42 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:43.070623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.142747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661584d8-0fe4-4077-9345-e5e88b507d92 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.143764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661584d8-0fe4-4077-9345-e5e88b507d92 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.144489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661584d8-0fe4-4077-9345-e5e88b507d92 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.153614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 116/214] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:43.156076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f8d9-df40-4c51-ab9d-62644a32cc7a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.156692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f8d9-df40-4c51-ab9d-62644a32cc7a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.157490 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292f8d9-df40-4c51-ab9d-62644a32cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.179694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a292f8d9-df40-4c51-ab9d-62644a32cc7a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:43.180948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:43.182148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a292f8d9-df40-4c51-ab9d-62644a32cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:43.207989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338ef-d4ca-4dcf-a670-1e4bac1d57e0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.261072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:43.267253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.268591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.268877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.269154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.270580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:43.272513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:43.272692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 117/215] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:43.273786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af541d-0d61-4ea5-9f48-d7cc1de0f1a9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.274067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af541d-0d61-4ea5-9f48-d7cc1de0f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.274343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af541d-0d61-4ea5-9f48-d7cc1de0f1a9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.275907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 118/216] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:43.278165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 99/217] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:43 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:43.288085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.289792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.290093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.290918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.310520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:43.311750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:43.404141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 119/218] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:43.405596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.462420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bce3db-d401-4b18-adf2-d255dd089ba5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.533045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.534642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.535104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.535559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/20428d815d0d44b28a7d873242e06a9c/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.539089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, user_id=20428d815d0d44b28a7d873242e06a9c, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:43.541840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:43.545532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241446-5c14-42eb-96ba-fb9d0a6defe3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.561387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 100/219] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:43 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/20428d815d0d44b28a7d873242e06a9c/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:43.570988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.594415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:43.595258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:43.624321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.626693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.627185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.627663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.630878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:43.635316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:43.645866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 120/220] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:43.648341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:43.786974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1be1-3598-4778-ac36-f40cde5feec0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.809723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b91f2-2ab8-4ce1-94c9-5e643de06885 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:43.840288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.841685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.842332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.842830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.843798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:43.846533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:43.884562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:43.885046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 121/221] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:43 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:07:43.886082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:43.886598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:43.887100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:43.911003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:43.936487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:43.938430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:43.940836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 101/222] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:43.948483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:44.064769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:44.188060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad268-0e79-4c7b-bf97-361e3028cf08 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:44.263360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:44.264082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:44.264365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:44.264636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:44.266111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:44.266594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:44.269090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:44.285610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 122/223] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:44.291397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:44.435389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:44.509675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b92d-c296-448c-ad1d-8bfcc797abde None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:44.579010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:44.580661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:44.581140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:44.581640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] PATH_INFO: `/v3/users/c5498b496ec743f69523ca9ceca138ff` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:44.607356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=c5498b496ec743f69523ca9ceca138ff)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:44.609921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04e209-7d8c-4d2b-b561-5493574d8ae5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:44.613264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 102/224] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:44 2022] GET /identity/v3/users/c5498b496ec743f69523ca9ceca138ff => generated 297 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:44.621890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:44.826909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee413ba9-7445-49be-bcea-b1ff7423948b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:44.849071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:44.854215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5e68-abd9-4002-afec-b9c14011bda6 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:44.854697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5e68-abd9-4002-afec-b9c14011bda6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:44.855181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d5e68-abd9-4002-afec-b9c14011bda6 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:44.858200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 103/225] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:44.870267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:44.870907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:44.871492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:44.911159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:44.912409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:44.915250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:44.915942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:44.916228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:44.916505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:45.012686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:45.014493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:45.167883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 123/226] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:45.176881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:45.184073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 104/227] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:45.204419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:45.223849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:45.400642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:45.525665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f8cf-690b-40a1-9cec-d8c7456c1d66 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:45.568626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b3be-d983-45d1-ae3f-42004640c1f3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:45.609806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:45.611390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:45.611880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:45.612359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:45.614867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:45.617808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:45.621154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 124/228] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:45 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:45.635219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:45.680961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:45.682371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:45.682858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:45.683347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:45.739156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:45.764370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:45.766177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:45.767737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 105/229] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 545 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:45.783054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:45.806904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:45.851349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6623e-45ed-4c14-974a-3ccc39ba2310 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:45.930488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:45.931111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:45.931401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:45.931715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:45.932767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=c5498b496ec743f69523ca9ceca138ff) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:45.933087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:45.934704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:45.946271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 125/230] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:45 2022] GET /identity/v3/role_assignments?user.id=c5498b496ec743f69523ca9ceca138ff&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:46.079648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1095-ca85-43b8-8fc9-de1f9f91ace5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:46.130664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:46.182810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:46.184567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:46.185402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:46.186185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:46.188011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:46.188872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:46.191924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:46.216361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 106/231] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:46.225306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:46.450795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89436-f3b7-476a-83a9-5d1e823b916e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:46.539376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:46.541464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:46.542305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:46.543060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:46.545786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:46.548986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:46.552568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 126/232] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:46 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:46.561032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:46.834302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfab8-7392-46a1-b966-7cd32088e3b6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:46.922816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:46.924522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:46.925022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:46.925567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:46.930913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:46.933720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1174cc7-1300-4664-86cf-4006509b1eb2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:46.937676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 107/233] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:46 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:46.946945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:46.974641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:47.103494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94798da6-fc97-4e3e-b022-ab634b104499 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:47.152978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:47.153956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.154340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.154625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/20428d815d0d44b28a7d873242e06a9c/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.161483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, user_id=20428d815d0d44b28a7d873242e06a9c, role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:47.164393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6236132-b687-4003-b567-fd03d353b991 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:47.197771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 127/234] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:46 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/20428d815d0d44b28a7d873242e06a9c/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:47.217911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6fd4a-0171-4d29-9677-6cd7d6454f83 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.218312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6fd4a-0171-4d29-9677-6cd7d6454f83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.218756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6fd4a-0171-4d29-9677-6cd7d6454f83 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.220681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 108/235] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:47.232723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.233192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.233733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.274536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:47.276787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:47.453997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:47.464133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 128/236] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:47.491361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6fd4a-0171-4d29-9677-6cd7d6454f83 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:47.561893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:47.655216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6fd4a-0171-4d29-9677-6cd7d6454f83 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:47.719548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:47.720464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.720761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.721041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.735574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f87a7c2f-7ff4-4bf8-a508-b03cbea75c6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:47.744283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:47.745878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:07:47.747317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:47.751825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 109/237] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:47.754127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:47.856505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593357e7-0314-430c-89c2-7befdd1c27da None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.857383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593357e7-0314-430c-89c2-7befdd1c27da None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.857540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593357e7-0314-430c-89c2-7befdd1c27da None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.860218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 110/238] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:47.872057 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206ff3-7208-4cd0-9bde-669ada8f28a2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:47.877896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206ff3-7208-4cd0-9bde-669ada8f28a2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:47.878851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206ff3-7208-4cd0-9bde-669ada8f28a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:47.927128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-dd206ff3-7208-4cd0-9bde-669ada8f28a2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:47.928946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd206ff3-7208-4cd0-9bde-669ada8f28a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:47.995786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da891dd-7cff-4ff0-88d4-9a403d68fc8b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.066792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.068211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 111/239] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:48.068841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.069740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.070490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba15bf-fbc1-49ec-82c1-495656054336 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.070906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.071056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba15bf-fbc1-49ec-82c1-495656054336 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.071345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba15bf-fbc1-49ec-82c1-495656054336 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.072885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:48.073746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:48.076917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:48.079615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 112/240] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:48.083531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.084044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.084525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.096834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 129/241] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:48.105377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.123970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:48.126034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:48.196385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:48.237567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:48.280195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 113/242] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:48.282103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.287433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deecdf-4e4d-4749-b2e1-c2a1c756d7f6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.331402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.332613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.333225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.333840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.355943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:48.365093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:48.366092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:48.366926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:48.369740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 130/243] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:48.377595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.519763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89ddf7-967d-4cc4-b86f-880eb899edd9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.521591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ad6c-dc7f-43a5-b8b8-39e610efae64 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.551385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:48.567914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.569140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.569722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.570212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.571968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:48.574353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:48.581527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 131/244] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:48.582835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.593374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.594776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.595268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.595770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.598279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:48.601225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:48.605085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 114/245] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:48.606507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.679058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:48.715167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d067da-4c15-455c-be3a-10f16f657f35 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.756616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.758167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.758739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.759232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.760307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:48.760902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:48.762719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:48.778257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 132/246] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:48.779670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.812788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7faec-f268-4aa4-af98-0a1df70a7e15 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.846503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:48.889487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.891948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.892893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.893760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:48.895309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:48.898780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:48.912730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216395e2-cd92-455a-a910-60ac8a1e6405 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:48.940662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 115/247] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:48 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:48.943162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:48.986608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:48.987546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:48.987865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:48.988208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:49.192055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:49.193783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:49.243685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:49.256492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686de1d-ddcc-41fc-9b33-ab78311d3a91 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:49.333743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:49.336107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:49.336856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:49.337633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] PATH_INFO: `/v3/users/ada578be3902475a9243028bea752cb3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:49.347903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:49.380169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] RBAC: Authorizing `identity:get_user(user_id=ada578be3902475a9243028bea752cb3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:49.383978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:49.388317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 116/248] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/users/ada578be3902475a9243028bea752cb3 => generated 297 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:49.395161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:49.433804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 133/249] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:49.440621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:49.646228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3505f14-400e-493c-af5a-ef70ed1403a7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:49.666337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0822b-5683-4fa8-ac52-2207638f815e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:49.718659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:49.721549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:49.722049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:49.722535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:49.726051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:49.726223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:49.727597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:49.727892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:49.728181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:49.728821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85e81d-fb53-40b1-8631-a124d207f3d8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:49.735909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 134/250] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:49 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:49.742671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a2a14-615e-4cfe-90a2-5eb67a570be0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:49.743183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a2a14-615e-4cfe-90a2-5eb67a570be0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:49.744724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a2a14-615e-4cfe-90a2-5eb67a570be0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:49.748498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 135/251] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:49.753968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a2a14-615e-4cfe-90a2-5eb67a570be0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:49.786525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:49.808720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:49.963181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:49.966071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:49.973740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a2a14-615e-4cfe-90a2-5eb67a570be0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:50.068605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:50.069844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.070465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.071004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.073063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=20428d815d0d44b28a7d873242e06a9c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:50.073651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:50.076340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf06e7-c20a-4e45-9e80-0b72c9229a27 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:50.095646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 136/252] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:49 2022] GET /identity/v3/role_assignments?user.id=20428d815d0d44b28a7d873242e06a9c&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:50.097441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100f9f-a70a-4d93-a52c-d0dc2b361eb1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.097970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100f9f-a70a-4d93-a52c-d0dc2b361eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.098460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100f9f-a70a-4d93-a52c-d0dc2b361eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.150762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-08100f9f-a70a-4d93-a52c-d0dc2b361eb1 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:50.152125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-08100f9f-a70a-4d93-a52c-d0dc2b361eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:50.158105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 117/253] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:50.165721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:50.282141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:50.364036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 137/254] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:50.365317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe06a0-628a-415a-84cc-ef07090fd67a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.368246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe06a0-628a-415a-84cc-ef07090fd67a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.368589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe06a0-628a-415a-84cc-ef07090fd67a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.370365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 138/255] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:50.373363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fe077-9fc9-475f-978d-06761d5d717c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:50.377158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.377474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.378862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.418425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:50.420617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:50.440705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:50.454068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:50.455170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.455459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.455741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.457467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:50.459063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:50.461509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 118/256] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:50 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:50.463331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:50.561097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 139/257] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:50.567021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:50.617622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c662de1-a2cf-4807-b03b-4bc14feee85d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:50.678635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:50.717041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:50.718195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.718451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.718729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.734711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:50.744321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:50.746103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:50.747785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 119/258] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:50.749107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:50.791121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a779107-e76e-4ac6-bf01-65c0b0a04e20 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:50.859144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:50.861748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:50.862279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:50.862777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:50.864510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=ada578be3902475a9243028bea752cb3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:50.865095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:50.867946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:50.879795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 140/259] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:50 2022] GET /identity/v3/role_assignments?user.id=ada578be3902475a9243028bea752cb3&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:50.881082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:50.895657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:50.924417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551091-9f7c-4ff2-85af-d596f98a735f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.026014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.027915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.028469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.028991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.031473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb73089-a7bb-4a18-b7f7-f8d9739fcbf6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.061080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:51.076301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:51.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:07:51.078260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:51.087643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 120/260] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:07:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:51.094534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.134838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.136428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.141911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.142291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.143051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:51.143411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.144961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.164395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 141/261] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:51.169312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.172036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:51.262578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce05383-ec23-4e7d-a6e4-c36604b5b7cd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.286272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:51.326561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda7728-fd9e-48c8-b75e-deaa3460bb24 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.338427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.338930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.339223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.339496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.340395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:51.340715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.342219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.362188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 121/262] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:07:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:51.370773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.379753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.381865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.385851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.386434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] PATH_INFO: `/v3/users/c5498b496ec743f69523ca9ceca138ff` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.389355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] RBAC: Authorizing `identity:get_user(user_id=c5498b496ec743f69523ca9ceca138ff)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.391375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.398036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 142/263] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:51 2022] GET /identity/v3/users/c5498b496ec743f69523ca9ceca138ff => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:51.401892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.568661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1c53-c9db-4c19-bee7-3ab88e4695dd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.573769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:51.607385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac1f7-37ef-4c16-8f34-0a3f39646914 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.671939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.673308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.674110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.674388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] PATH_INFO: `/v3/users/20428d815d0d44b28a7d873242e06a9c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.676101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] RBAC: Authorizing `identity:get_user(user_id=20428d815d0d44b28a7d873242e06a9c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.677721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.679694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 122/264] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:51 2022] GET /identity/v3/users/20428d815d0d44b28a7d873242e06a9c => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:51.681598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.682497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.682778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.683058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.684375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.686136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.688388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 143/265] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:51 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:51.689165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.694994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:51.705185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:51.887424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06409-f5f4-4219-a9af-9f5d39998c61 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.950159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b933e-2ef9-447e-b856-8bb25da826f6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:51.957620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:51.961369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:51.961882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:51.962377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/c5498b496ec743f69523ca9ceca138ff/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:51.973372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, user_id=c5498b496ec743f69523ca9ceca138ff, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:51.973372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802fbe6-afad-4755-8f2f-f4e62a0d3d47 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:51.999954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 144/266] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:51 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/c5498b496ec743f69523ca9ceca138ff/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:52.033393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:52.034769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.035266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.035721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.095575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:52.164367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:52.166224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:52.317845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d116-4aed-4e2e-8da0-fdd8829b4cab None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.318456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d116-4aed-4e2e-8da0-fdd8829b4cab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.318561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 123/267] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:52.319557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01d116-4aed-4e2e-8da0-fdd8829b4cab None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.326949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 145/268] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:52.327971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:52.334545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b94c63-2379-4c1e-b577-d28d24f160f2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.334911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b94c63-2379-4c1e-b577-d28d24f160f2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.335123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b94c63-2379-4c1e-b577-d28d24f160f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.371223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-04b94c63-2379-4c1e-b577-d28d24f160f2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:52.372862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04b94c63-2379-4c1e-b577-d28d24f160f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:52.466260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:52.479839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 146/269] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:52.483077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20047b-03a7-41dd-adc1-54043eca82c2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.483829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20047b-03a7-41dd-adc1-54043eca82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.484601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20047b-03a7-41dd-adc1-54043eca82c2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.487099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 147/270] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:52.487596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:52.496609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.497060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.497559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.526528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15f884-22c9-440f-a92c-dcd0863f601a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:52.530339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:52.532394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:52.568351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:52.569146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.569605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.569896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.571306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:52.572883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:52.575184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 124/271] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:52 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:52.576060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:52.682324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 148/272] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:52.691045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:52.705629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e25e1c-73b0-4c47-baff-ea6107acdb57 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:52.753306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:52.754252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.754537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.754872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.761953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:52.772489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:52.783191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:52.785388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 125/273] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:52.787198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:52.903202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6eb37-1a87-471e-9c56-363635c2f6b2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:52.944483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:52.958625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:52.959447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:52.959741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:52.960020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:52.961653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=20428d815d0d44b28a7d873242e06a9c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:52.961993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:52.964034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:52.974170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 149/274] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:52 2022] GET /identity/v3/role_assignments?user.id=20428d815d0d44b28a7d873242e06a9c&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:52.975115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:52.991290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e8db-1452-45c7-a92e-2052e39c219e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:53.113679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:53.115256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:53.115741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:53.116289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:53.121899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d837-74d2-4a57-8be1-f80e38b64b60 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:53.125251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:53.128034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:53.131502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 126/275] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:53.140666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:53.178800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:53.180316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:53.180615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:53.180904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:53.181798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:53.182118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:53.184202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:53.196971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 150/276] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:53.204986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:53.312807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:53.368491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:53.373945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4780e-5872-4144-bc46-103ebd2acbfb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:53.449378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:53.450613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:53.451099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:53.451719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:53.453226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:53.455911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:53.456176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326672c-f870-4aea-a24e-aedcb56eefee None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:53.498573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 127/277] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:07:53 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:53.509974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:53.510933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:53.511272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:53.511623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] PATH_INFO: `/v3/users/1549b9bff4184119ac0f7d1bff8d3239` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:53.532436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] RBAC: Authorizing `identity:get_user(user_id=1549b9bff4184119ac0f7d1bff8d3239)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:53.534312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9bf32-9e7b-4a66-ad7e-092554f086fe None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:53.536775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 151/278] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:53 2022] GET /identity/v3/users/1549b9bff4184119ac0f7d1bff8d3239 => generated 293 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:53.544821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:53.694800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f9ed-07a8-4242-a5e4-2450b9b07522 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:53.720539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:53.749084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:53.749929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:53.750221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:53.750511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:53.751471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, user.id=1549b9bff4184119ac0f7d1bff8d3239, scope.system=all) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:53.751859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:53.753436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b802-5e7f-4023-bca5-6b7b319d149f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:53.763097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 128/279] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 25 16:07:53 2022] GET /identity/v3/role_assignments?user.id=1549b9bff4184119ac0f7d1bff8d3239&scope.system=all&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:54.045764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9516c3-c55e-45a3-b5b6-a589707eb23d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:54.046238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9516c3-c55e-45a3-b5b6-a589707eb23d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:54.046377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9516c3-c55e-45a3-b5b6-a589707eb23d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:54.047886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 152/280] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:54.057237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:54.057720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:54.058202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:54.090922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:54.093022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:54.224423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 129/281] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:54.260146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9516c3-c55e-45a3-b5b6-a589707eb23d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:54.453989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9516c3-c55e-45a3-b5b6-a589707eb23d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:54.481932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:54.515601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:54.516539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:54.516698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:54.517023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:54.539248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:54.547272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:54.551347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:54.552842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 153/282] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:54.564221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:54.800737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3027-b305-482f-99a6-f94ddd761e9d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:54.827837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:54.868462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:54.870295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:54.871114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:54.871942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:54.873838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:54.874726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:54.877799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06542231-757c-4ad3-ae92-d32ef822e142 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:54.899947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 130/283] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:54.908633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.051217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60166-f7b4-4067-a10f-33fdca590beb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:55.051949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830885-a54d-442c-afd9-93848637b2c0 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.052245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830885-a54d-442c-afd9-93848637b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.052526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830885-a54d-442c-afd9-93848637b2c0 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.054372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 131/284] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:55.060735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.061130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.061454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.090287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:55.092688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:55.122892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:55.124592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.125093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.125612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] PATH_INFO: `/v3/users/ada578be3902475a9243028bea752cb3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.134664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] RBAC: Authorizing `identity:get_user(user_id=ada578be3902475a9243028bea752cb3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:55.136540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:55.139770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 154/285] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:54 2022] GET /identity/v3/users/ada578be3902475a9243028bea752cb3 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:55.145544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.200730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 132/286] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:55.225613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.233801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:55.359166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671362e-f111-4596-81f5-8dd48a75cc01 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:55.365740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:55.366024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf004d-fa69-42c1-80a5-2c18e855e957 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:55.444117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:55.446983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.448361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.449303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.453871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:55.455144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.455576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.455988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.458893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:55.460582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:55.466466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 155/287] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:55 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:55.471901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.477791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:55.487261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:55.488411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:55.489471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:55.506702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 133/288] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:07:55.511136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.690816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc995f-c174-48cc-bc8e-96d014116d0e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:55.700161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:55.736869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:55.738162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.738580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.739126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/ada578be3902475a9243028bea752cb3/roles/01fc9b48f93348c88c5cb563a080e941` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.741920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, user_id=ada578be3902475a9243028bea752cb3, role_id=01fc9b48f93348c88c5cb563a080e941)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:55.743876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:55.759404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab999325-69c2-4c77-841a-e6afc4d86b90 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:55.772875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 156/289] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:07:55 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/users/ada578be3902475a9243028bea752cb3/roles/01fc9b48f93348c88c5cb563a080e941 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:07:55.804743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:55.805849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:55.806279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:55.806764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:55.807762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:55.808216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:55.810008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:55.821418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 134/290] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:55.827795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:55.981783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f59b-662b-4b1e-8406-f71ad7eb1ff7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:56.039071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:56.039993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.040366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.040647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] PATH_INFO: `/v3/users/c5498b496ec743f69523ca9ceca138ff` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.042082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] RBAC: Authorizing `identity:get_user(user_id=c5498b496ec743f69523ca9ceca138ff)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:56.043671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:56.045478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 157/291] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:55 2022] GET /identity/v3/users/c5498b496ec743f69523ca9ceca138ff => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:56.051604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:56.097935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:56.233919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729f550-a4fd-474a-828d-22ebae9c4388 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:56.297646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:56.298434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.298888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.299177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.415380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:56.417050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11623a6f-5224-4af3-bfb2-09c7c756f351 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:56.470787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:56.566728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 135/292] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:07:56.574827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:56.650647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:56.735183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e60c-ad42-43b9-a6f4-c07710b913df None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.735796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e60c-ad42-43b9-a6f4-c07710b913df None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.736329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e60c-ad42-43b9-a6f4-c07710b913df None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.738720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 136/293] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:56.752497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efa5-ed7a-4cba-8554-e00c1a095b68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.752814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efa5-ed7a-4cba-8554-e00c1a095b68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.753070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efa5-ed7a-4cba-8554-e00c1a095b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.781144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0db57-d23a-47b5-8617-bb6b26ccceb2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:56.789077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-98f5efa5-ed7a-4cba-8554-e00c1a095b68 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:56.790727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-98f5efa5-ed7a-4cba-8554-e00c1a095b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:56.854015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:56.855467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.856223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.856893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.859718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:56.862977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ee1ed-29e0-4b73-8736-228b68457e43 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:56.867422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 158/294] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:07:56 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:56.869916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45717bad-cc9c-4f08-9ef9-471878abb44f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.870435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45717bad-cc9c-4f08-9ef9-471878abb44f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.870922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45717bad-cc9c-4f08-9ef9-471878abb44f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.874131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 159/295] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:56.880312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45717bad-cc9c-4f08-9ef9-471878abb44f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:56.907109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 137/296] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:56.910230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:56.911072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:56.911823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:56.948317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:56.951035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:57.025723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.051120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 138/297] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:57.052445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.101962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45717bad-cc9c-4f08-9ef9-471878abb44f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.105927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.177619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.178878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.179306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.179823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.181505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=c5498b496ec743f69523ca9ceca138ff) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:57.182085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:57.185038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03a67a-fd39-4df7-9828-0b7b0eb79e4f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:57.201391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 160/298] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:07:56 2022] GET /identity/v3/role_assignments?user.id=c5498b496ec743f69523ca9ceca138ff&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:57.201391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d859de-2d7e-4485-a476-798b67fd0207 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.201391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d859de-2d7e-4485-a476-798b67fd0207 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.201391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d859de-2d7e-4485-a476-798b67fd0207 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.204749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 161/299] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:07:57.204749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d859de-2d7e-4485-a476-798b67fd0207 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.212841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b80d-77d6-4237-a6de-88b14cb9a2aa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.285735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.289582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.290389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.291349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.294561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:57.299274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c9ac-c6cd-418a-9321-7108a1743dd1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:57.304903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 139/300] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:07:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:57.309884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.311046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.312468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.357406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:57.360560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:57.383778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d859de-2d7e-4485-a476-798b67fd0207 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.446885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.448251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.448754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.449035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.455121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.476503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:57.486900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:57.487937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:07:57.488843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:57.490170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 162/302] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:57.490276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 140/302] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:07:57.493428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.499475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.550569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.652488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01605bf-c50e-49fe-a8cf-c2fa9c26916c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.653304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455fe90-1355-4d88-aeb1-36a1a5aadc23 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.697764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.698456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.698752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.699110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] PATH_INFO: `/v3/groups` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.700251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:57.701844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:57.704858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.706070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.706647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.707180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:57.709006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:57.709320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:57.712577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:57.729407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 163/303] 10.222.0.66 () {60 vars in 1134 bytes} [Mon Jul 25 16:07:57 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:07:57.729407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.744136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 141/304] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:57.747923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:57.865903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.884607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:57.905925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87229574-321f-4d8a-b014-5ec0d65864a5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.958980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd44ea-df7e-4a0a-8ab4-be454ccae9a3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:57.983264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:57.984470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:57.986471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:57.986471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.004835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:58.006641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.007833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.008439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] PATH_INFO: `/v3/users/1549b9bff4184119ac0f7d1bff8d3239` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.008698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:58.010413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] RBAC: Authorizing `identity:get_user(user_id=1549b9bff4184119ac0f7d1bff8d3239)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:58.012235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:58.014273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 142/305] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:57 2022] GET /identity/v3/users/1549b9bff4184119ac0f7d1bff8d3239 => generated 293 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:58.017387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:58.018544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:58.019654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:07:58.021437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 164/306] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:58.023378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:58.025549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:58.206695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b2e-9ad9-45d7-b579-287b061f115d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:58.260164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039fe52-b19b-4714-93ee-142c9bb652f6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:58.286973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:58.288642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.289249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.289824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.291364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:58.294031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:58.294986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:58.298994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:58.321828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 165/307] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:58.328379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:58.332656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:58.334528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.335223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.335926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] PATH_INFO: `/v3/system/users/1549b9bff4184119ac0f7d1bff8d3239/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.339357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1549b9bff4184119ac0f7d1bff8d3239, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:58.342220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:58.354847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:58.371558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 143/308] 10.222.0.66 () {58 vars in 1334 bytes} [Mon Jul 25 16:07:58 2022] PUT /identity/v3/system/users/1549b9bff4184119ac0f7d1bff8d3239/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:07:58.567625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3728-51fd-4838-a51a-0f0cfeb519b4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:58.636972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:58.639354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.640197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.640972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] PATH_INFO: `/v3/users/5ea3d5a44da54200ba5865be572f6e03` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.671412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] RBAC: Authorizing `identity:get_user(user_id=5ea3d5a44da54200ba5865be572f6e03)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:58.674690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f38ad-ea54-40ef-a403-cebd5cb6bb70 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:58.678851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 166/309] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:58 2022] GET /identity/v3/users/5ea3d5a44da54200ba5865be572f6e03 => generated 293 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:07:58.685568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:58.810946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b0f0-f259-4ab1-b0f1-aec06637b8f3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.811482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b0f0-f259-4ab1-b0f1-aec06637b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.811988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b0f0-f259-4ab1-b0f1-aec06637b8f3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.821579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 167/310] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:07:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:07:58.824229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.824732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.825222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.860104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:07:58.862310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:07:58.886965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0262de-c203-4cb3-b1ca-8a385265abea None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:58.974659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:58.977367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:58.978202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:58.978957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:58.980878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, user.id=5ea3d5a44da54200ba5865be572f6e03, scope.system=all) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:58.982806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:58.985929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:58.996916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:59.002102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 144/311] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 25 16:07:58 2022] GET /identity/v3/role_assignments?user.id=5ea3d5a44da54200ba5865be572f6e03&scope.system=all&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:59.017264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 168/312] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:07:59.041278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:59.130978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:59.212019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfd40a-df9e-4c48-894e-c5a149ef6d18 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:59.253261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:59.254142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:59.254429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:59.254710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:59.274318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:07:59.285496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:07:59.286934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:07:59.287777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 145/313] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:07:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:07:59.295596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:59.496927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8fdc9-0d0c-4792-bb34-970a4384b3ea None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:59.566440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:59.567639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:59.568128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:59.568618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:59.569899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:07:59.570559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:59.573307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:59.590929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 169/314] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:07:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:59.596957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:59.780645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163582d4-d46f-4067-b4cc-39eb6df640ab None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:07:59.783226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:07:59.824817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:07:59.825780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:07:59.826066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:07:59.826345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] PATH_INFO: `/v3/users/ada578be3902475a9243028bea752cb3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:07:59.828129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] RBAC: Authorizing `identity:get_user(user_id=ada578be3902475a9243028bea752cb3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:07:59.829690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46429f-c8bd-4ec7-afe4-49793f3ac5bd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:07:59.836721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 146/315] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:07:59 2022] GET /identity/v3/users/ada578be3902475a9243028bea752cb3 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:07:59.838900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:07:59.961292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:00.049588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e3c82-0db6-4704-8794-a85db4f8cbf8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:00.116119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:00.116812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.117144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.117608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.233743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:00.237465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:00.254728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aa918-84d0-48e5-b6a2-b9551222481b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.255079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aa918-84d0-48e5-b6a2-b9551222481b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.255281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aa918-84d0-48e5-b6a2-b9551222481b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.261394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 147/316] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:00.264416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.265024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.265619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.287541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:00.289291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:00.323241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:00.373851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 148/317] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:00.388441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 170/318] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:00.390937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:00.400862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:00.565900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:00.570022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a9d3-e451-426b-938d-0fe910ebcc62 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:00.590895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd4aa4-6df7-4c6f-a4f9-1e9d841bb718 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:00.613172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:00.615066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.615499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.616227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.618831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:00.621673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:00.624266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 149/319] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:00 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:00.633373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:00.654042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:00.655611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.656015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.656513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.659560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:00.662904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:00.669424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 171/320] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:00.674587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:00.737886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:00.792989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1142-1feb-4750-a38d-6d3f3be2bdef None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:00.845279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:00.846337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.846802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.847254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.849297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, scope.project.id=6ae85123e50c4564bc6f70df335004ad, user.id=ada578be3902475a9243028bea752cb3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:00.849731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:00.851614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c5e44-8e74-4c7e-8505-164a7330f703 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:00.869481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 150/321] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:00 2022] GET /identity/v3/role_assignments?user.id=ada578be3902475a9243028bea752cb3&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:00.872349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01f9b-7efa-47ee-948a-3adcba4009a5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:00.923528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:00.924721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:00.925289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:00.925830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] PATH_INFO: `/v3/groups` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:00.926882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:00.928832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd375c-db60-454d-8c62-99d50b2bae69 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:00.951471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 172/322] 10.222.0.66 () {60 vars in 1134 bytes} [Mon Jul 25 16:08:00 2022] POST /identity/v3/groups => generated 226 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:01.174818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ea03-2dfd-424c-ab56-04f180289677 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.175511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ea03-2dfd-424c-ab56-04f180289677 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.175799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573ea03-2dfd-424c-ab56-04f180289677 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.177551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 151/323] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:01.183797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c46f0-f70c-4a7a-ae8f-f78a8f731a38 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.184375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c46f0-f70c-4a7a-ae8f-f78a8f731a38 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.184375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c46f0-f70c-4a7a-ae8f-f78a8f731a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.205657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ac5c46f0-f70c-4a7a-ae8f-f78a8f731a38 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:01.207119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ac5c46f0-f70c-4a7a-ae8f-f78a8f731a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:01.328757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 173/324] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:01.354081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573ea03-2dfd-424c-ab56-04f180289677 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:01.385734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5210eb-689b-45b0-9923-33abeff545ce None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.386013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5210eb-689b-45b0-9923-33abeff545ce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.386284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5210eb-689b-45b0-9923-33abeff545ce None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.388122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 174/325] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:01.393838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.394265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.394430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.411502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:01.415404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:01.416669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:01.433062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:01.523020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 175/326] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:01.549121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:01.584372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573ea03-2dfd-424c-ab56-04f180289677 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:01.658695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:01.660242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.660720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.661204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.673218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9076011-8d5b-4926-af01-eec82399e1de None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:01.683723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:01.700172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:01.701664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:01.703236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:01.709764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 152/327] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:01.713154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:01.715864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:01.716819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:01.717153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:01.717458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:01.735107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:01.744299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:01.746559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:01.748449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 176/328] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:01.753217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:01.775617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:01.940072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:01.956513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf1bfb-4869-418d-b010-c5424eb5c335 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:01.978937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b8c3b-cac4-4852-8324-470cd5abe2ef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:02.019990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.021230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.021756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.022234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.023343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:02.023899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.026371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.040737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 153/329] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:02.044098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:02.051482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.052614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.053055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.053524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.054978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:02.055498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.057944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.077932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 177/330] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:02.089793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:02.228558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3555e-79c3-4b83-a57b-bd55920d4bd8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:02.278042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:02.344958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe50a59-60ad-4a66-9e0a-ff9d65d493af None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:02.358596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.360752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.361578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.362290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] PATH_INFO: `/v3/users/1549b9bff4184119ac0f7d1bff8d3239` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.367639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] RBAC: Authorizing `identity:get_user(user_id=1549b9bff4184119ac0f7d1bff8d3239)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.370864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.378956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 154/331] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:08:02 2022] GET /identity/v3/users/1549b9bff4184119ac0f7d1bff8d3239 => generated 293 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:02.394798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:02.431031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.432991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.433512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.433970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] PATH_INFO: `/v3/users/5ea3d5a44da54200ba5865be572f6e03` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.439841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] RBAC: Authorizing `identity:get_user(user_id=5ea3d5a44da54200ba5865be572f6e03)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.446361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.446943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 178/332] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:08:02 2022] GET /identity/v3/users/5ea3d5a44da54200ba5865be572f6e03 => generated 293 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:02.456542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:02.479947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:02.688078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d48e-26d0-4520-9c0c-f5f8e00413b8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:02.727935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300e1ac-a56d-4506-87c4-8c66803bdd27 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:02.797535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.798705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.799211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.799673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] PATH_INFO: `/v3/system/users/5ea3d5a44da54200ba5865be572f6e03/roles/01fc9b48f93348c88c5cb563a080e941` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.801625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ea3d5a44da54200ba5865be572f6e03, role_id=01fc9b48f93348c88c5cb563a080e941)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.803542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e5767-a7b6-46e4-a9af-fe746f8c2afb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.812936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:02.814786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:02.816251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:02.816774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:02.817272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:02.818599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 179/333] 10.222.0.66 () {58 vars in 1334 bytes} [Mon Jul 25 16:08:02 2022] PUT /identity/v3/system/users/5ea3d5a44da54200ba5865be572f6e03/roles/01fc9b48f93348c88c5cb563a080e941 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:02.819604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] RBAC: Adding query filter params (role.id=c618a32897ba4eb08e80389e7b3d1cc9, user.id=1549b9bff4184119ac0f7d1bff8d3239, scope.system=all) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:02.820201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:02.824177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d064-84cf-411d-b72b-12d5c9e11482 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:02.866971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 155/334] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 25 16:08:02 2022] GET /identity/v3/role_assignments?user.id=1549b9bff4184119ac0f7d1bff8d3239&scope.system=all&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 510 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:04.628978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fffb0-2401-4d58-a60b-b59c975ce6d1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:04.629770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fffb0-2401-4d58-a60b-b59c975ce6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:04.630040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fffb0-2401-4d58-a60b-b59c975ce6d1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:04.632309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 180/335] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:04.642447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:04.643199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:04.643199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:04.665184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:04.666616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:04.780608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 156/336] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:04.820609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fffb0-2401-4d58-a60b-b59c975ce6d1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:05.063202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fffb0-2401-4d58-a60b-b59c975ce6d1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:05.139712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:05.141831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:05.142640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:05.143410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:05.171243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:05.191574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:05.193210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:05.194776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:05.199636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 181/337] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:05.205574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:05.231601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:05.376045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a9e8-05d5-4cb5-ab37-8c7208d9cadf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:05.427322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:05.428031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:05.428382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:05.428689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:05.429609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:05.430012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:05.431503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beaa56-8294-4dc8-b320-57214fed73f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:05.444400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 157/338] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:05.449971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:05.574737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:05.593397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278049e-84d2-405f-9e77-139bfafeca2c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:05.593749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278049e-84d2-405f-9e77-139bfafeca2c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:05.593973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278049e-84d2-405f-9e77-139bfafeca2c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:05.596083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 158/339] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:05.604511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:05.604829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:05.605165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:05.656605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2772-601b-4438-a9bd-7a31a22e7ff6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:05.674474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:05.676393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:05.755817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:05.756928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:05.757894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:05.758383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:05.913731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 159/340] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:05.937420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:05.939523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:05.971244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:05.975266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:06.060888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:06.168681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 182/341] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:06.176860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:06.191517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6dc44-a0af-481a-9457-81039488fc6e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:06.312127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:06.314053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:06.314832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:06.315548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:06.348204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-19846431-5fc7-4aee-806f-5108efacc768 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:06.358459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:06.359746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:06.360759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 160/342] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:06.362841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7ed50-9eeb-459c-83ed-2b184ef04a50 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:06.363846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7ed50-9eeb-459c-83ed-2b184ef04a50 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:06.364213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7ed50-9eeb-459c-83ed-2b184ef04a50 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:06.365809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 161/343] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:06.370832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7ed50-9eeb-459c-83ed-2b184ef04a50 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:06.499795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dfa07-777f-47d1-acfc-15df101774d7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:06.568888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:06.577298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:06.578717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:06.579569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:06.580064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:06.582075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:06.584034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aabcea-8641-4b80-aa36-a7025b0ca413 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:06.588915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 183/344] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:06 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:06.591152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:06.591709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:06.593458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:06.623113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:06.625536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:06.676303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:06.679788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7ed50-9eeb-459c-83ed-2b184ef04a50 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:06.763400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:06.764568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:06.765066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:06.765621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:06.766712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:06.767313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:06.769052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:06.792597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 162/345] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:06.794145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:06.803120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 184/346] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:06.806361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:06.963338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fec88b-3e46-4efa-8139-3b1776fff2a2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:06.994983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:07.016806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:07.018252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.019147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.022773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.036820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:07.038985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:07.048731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 163/347] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:06 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:07.050006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:07.162445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79440a66-d7e3-4502-8cdd-c9c5a72ab368 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:07.190577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36786-cf3c-4b54-adb6-47eff00bfc85 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:07.240532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:07.241970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.242479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.242958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.269036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:07.278803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:07.279785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:07.280584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:07.283820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 164/348] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:07.285419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:07.286196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:07.287155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.287639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.288113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.354647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:07.395546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:07.425188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:07.427042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990cf9a-f8e0-4b37-8709-753292b272c6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:07.469379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1f87-e94a-4375-9bc0-ec95ab06bd9b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:07.523175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:07.524301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.524813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.525299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.527519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:07.528089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:07.535531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:07.561528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 165/349] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:07 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:07.563253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:07.567713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 185/350] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:07.569295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b0d60-add9-4a1a-8cdb-1c96511277c7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.569737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b0d60-add9-4a1a-8cdb-1c96511277c7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.570181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b0d60-add9-4a1a-8cdb-1c96511277c7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.571645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 186/351] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:07.581281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ab4-d418-41c1-af98-cf4c84c2878e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.581697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ab4-d418-41c1-af98-cf4c84c2878e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.582048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ab4-d418-41c1-af98-cf4c84c2878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.623929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-854b8ab4-d418-41c1-af98-cf4c84c2878e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:07.625316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-854b8ab4-d418-41c1-af98-cf4c84c2878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:07.743469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad79f3-0713-4201-b7d4-808f2d621dd3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:07.781979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:07.798054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:07.806188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:07.806995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.807279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.810434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.813252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:07.813594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:07.815206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:07.827074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 166/352] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:07.828272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:07.832225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 187/353] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:07.835448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9738a-574e-4154-a776-07f756e54faa None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:07.836390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9738a-574e-4154-a776-07f756e54faa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:07.837113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9738a-574e-4154-a776-07f756e54faa None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:07.840232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 188/354] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:07.842157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9738a-574e-4154-a776-07f756e54faa None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:07.985833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95421a-c1e5-44ec-b28e-41e3da3a535c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.017920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9738a-574e-4154-a776-07f756e54faa None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.037467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.038310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.038645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.038924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.040310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:08.041931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cece6fa-408a-416f-b0d4-8e68e0ec2bbb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:08.044198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 167/355] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:07 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:08.045451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13467033-4991-498d-a97d-b815a02a2372 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.045797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13467033-4991-498d-a97d-b815a02a2372 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.046072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13467033-4991-498d-a97d-b815a02a2372 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.047988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 168/356] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:08.049442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.049735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.050013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.072974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.073849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.074203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.074598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.074745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:08.076051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:08.122747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:08.153801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:08.174144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:08.175863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:08.177486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 169/357] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:08.178834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:08.325679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 189/358] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:08.326975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:08.380196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2b16d-86ff-4643-b9c1-5ab3bdf4c606 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.466411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.467207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.467432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.467808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.487083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:08.487283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-530dc775-d978-47d4-b87a-50d5d87abeaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:08.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:08.498863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:08.500336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:08.511190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 170/359] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:08.511190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.511190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.511190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.519426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e566796-6b77-486f-b8e1-9c13c3684932 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.540598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:08.542156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:08.598449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.601407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.602594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.603333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.605827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:08.606043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:08.612578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:08.613403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 190/360] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:08.615050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:08.698841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 171/361] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:08.700303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:08.828234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ea49-3c36-4e49-8583-5d09becc0ecc None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.879246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.880169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.880645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.880945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.900112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:08.910528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:08.912260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 16:08:08.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:08.918652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa1cae-6323-4749-a901-eb8fbfca8e96 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:08.919429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 191/362] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:08.919429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:08.956676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:08.982109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:08.983608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:08.984071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:08.984555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:08.986953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:08.991134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:08.996789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 172/363] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:08.998937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.050537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:09.101868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c05e3-a754-4d5b-b5ca-59c6a5527369 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.160906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.161646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.161942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.162222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.163203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:09.163371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.165117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:09.176412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 192/364] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:09.177535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.203509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210a3e3-d107-4dc2-94ee-4aa1ae515014 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.282786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.284525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.284977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.285484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.287187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=6ae85123e50c4564bc6f70df335004ad) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:09.287709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.292418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:09.302438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 173/365] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:08 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 260 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:09.303650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.334040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:09.377859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408c7de-5222-4329-97a1-7c37c373e666 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.432292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.437825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.438187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.441705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.464308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:09.476305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:09.477543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:09.478901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 193/366] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:09.481013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.499869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:09.513029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed319e9-06b5-4a25-a3cd-a7ecd65a49e8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.565178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.566331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.566633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.566914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.567882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:09.568237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.570919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:09.588626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 174/367] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:09.590139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.652826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77647-b888-46a3-88d7-1774cce69212 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.708979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.711163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.711691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.712170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.713844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.715631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:09.718426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 194/368] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:09.719606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.734669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:09.772870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8a40-c547-4298-90e3-76e0d7b026a2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.835209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.835209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.835209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.835701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.874438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0322a-47a2-45f1-adf7-f7a4f1547168 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:09.876445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:09.917665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd327689-c09c-4038-9165-88108594a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:09.918826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd327689-c09c-4038-9165-88108594a31c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:09.919334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd327689-c09c-4038-9165-88108594a31c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:09.919837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd327689-c09c-4038-9165-88108594a31c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:09.921038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd327689-c09c-4038-9165-88108594a31c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:09.921612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd327689-c09c-4038-9165-88108594a31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.923568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd327689-c09c-4038-9165-88108594a31c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:09.936409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 195/369] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:09.938519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd327689-c09c-4038-9165-88108594a31c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:09.947684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:09.949714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:10.060515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:10.090718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 175/370] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:10.090718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:10.125279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd327689-c09c-4038-9165-88108594a31c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:10.168053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:10.184949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:10.186667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:10.194504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:10.195147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] PATH_INFO: `/v3/users/5ea3d5a44da54200ba5865be572f6e03` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:10.199153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] RBAC: Authorizing `identity:get_user(user_id=5ea3d5a44da54200ba5865be572f6e03)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:10.202391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:10.206359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 196/371] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:08:09 2022] GET /identity/v3/users/5ea3d5a44da54200ba5865be572f6e03 => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:10.208150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:10.232025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347567f3-0ec7-4566-afaa-4e9db48e7c6b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:10.321827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:10.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:10.322745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:10.323076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:10.323994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:10.324390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:10.326081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:10.337394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 176/372] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:10 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:10.337394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:10.453653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ee356-0b28-4363-beaf-2400810496bf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:10.539062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:10.540034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:10.540445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:10.540822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:10.557969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:10.605773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82656f1b-1ad2-422b-b8d5-2b1b37b79454 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:10.655723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:10.698711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:10.703001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:10.703680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:10.704159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:10.706313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:10.709641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:10.713442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 177/373] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:10 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:10.714831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:10.715994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:10.718389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:11.032894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d59f-089c-452a-88a9-140939c5c802 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:11.078463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 197/374] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 872 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:11.081042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:11.122712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:11.123888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.124378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.124875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.126519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] RBAC: Adding query filter params (role.id=01fc9b48f93348c88c5cb563a080e941, user.id=5ea3d5a44da54200ba5865be572f6e03, scope.system=all) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:11.127029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:11.129108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:11.174855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 178/375] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 25 16:08:10 2022] GET /identity/v3/role_assignments?user.id=5ea3d5a44da54200ba5865be572f6e03&scope.system=all&role.id=01fc9b48f93348c88c5cb563a080e941 => generated 510 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:11.176861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:11.191031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:11.279181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:11.337712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828faa36-d4c2-48da-8174-e07c798afa8c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:11.385900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899383d3-a17b-411b-9c3f-50b63d29cde1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:11.460966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:11.463193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.463455 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:11.464263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.464878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.465273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] PATH_INFO: `/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.465454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.465818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.468429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] RBAC: Authorizing `identity:get_project(project_id=b8ee5c3834cf43d4a8d8dd7aee49ae24)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:11.471811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69badb-9ad0-466e-8ef5-1ba99fc58848 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:11.475207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:11.476512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 179/376] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:11 2022] GET /identity/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24 => generated 344 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:11.478216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:11.479557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ff86d-f89e-407b-b5bc-36cff34f117b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.480289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ff86d-f89e-407b-b5bc-36cff34f117b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.480995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ff86d-f89e-407b-b5bc-36cff34f117b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.484518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 180/378] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:11.484732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 198/378] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:11 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:11.491522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:11.494945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.495853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.496573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.530528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:11.533227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:11.671722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 181/379] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:11.673797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:11.716871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711d2c-f905-45a9-bfbf-cf7d7e605292 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:11.721971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:11.790188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:11.803321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:11.804961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:11.805498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:11.805991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] PATH_INFO: `/v3/groups/8b12503dafb14f098ed7b9ac60a43129` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:11.822261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] RBAC: Authorizing `identity:get_group(group_id=8b12503dafb14f098ed7b9ac60a43129)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:11.825439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:11.829060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 199/380] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:11 2022] GET /identity/v3/groups/8b12503dafb14f098ed7b9ac60a43129 => generated 229 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:11.830435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:11.910212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda36e8-43ed-49e7-83d3-892cb22f9880 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:12.002956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:12.004306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.004807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.005372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.007548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=6ae85123e50c4564bc6f70df335004ad) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:12.008153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:12.011032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:12.040615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 182/381] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:11 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 260 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:12.042295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:12.070567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2513-9efd-4c07-9ca1-3aa254ba4b86 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:12.149003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:12.150527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.151027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.151535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.200359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:12.219523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:12.221246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:12.222782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 200/382] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:12.233317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:12.319986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:12.340063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0bccb-e613-40f4-aa7f-acf9cc3fdd0e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:12.352104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:12.401456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:12.402227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.402554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.402855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.403729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] RBAC: Adding query filter params (group.id=8b12503dafb14f098ed7b9ac60a43129, role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=b8ee5c3834cf43d4a8d8dd7aee49ae24) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:12.404049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:12.405599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:12.421260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 183/383] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:12 2022] GET /identity/v3/role_assignments?group.id=8b12503dafb14f098ed7b9ac60a43129&scope.project.id=b8ee5c3834cf43d4a8d8dd7aee49ae24&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 260 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:12.490491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d661-2502-4c57-998b-9bbfc133d426 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:12.546605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:12.548257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.549035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.549850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.551600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:12.552534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:12.555525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764de61-982e-43ef-a070-5d73134fc70b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:12.577714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 201/384] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:12.586523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:12.591621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66021-ac4f-44b4-bc24-b2c3f9ae53e1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.592582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66021-ac4f-44b4-bc24-b2c3f9ae53e1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.593365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66021-ac4f-44b4-bc24-b2c3f9ae53e1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.596903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 202/385] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:12.603855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229af38-3847-4c65-8365-090be7becbaf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.604818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229af38-3847-4c65-8365-090be7becbaf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.605609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229af38-3847-4c65-8365-090be7becbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.647247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6229af38-3847-4c65-8365-090be7becbaf None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:12.649645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6229af38-3847-4c65-8365-090be7becbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:12.845096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 203/386] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:12.861864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb278a1c-6bc9-43d0-b28b-7c88bdd23d0e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:12.882956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229af38-3847-4c65-8365-090be7becbaf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:12.944901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:12.957800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:12.959407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:12.959900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:12.960438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:12.969331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:12.972185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:12.988769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 184/387] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:12 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:13.015553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:13.178101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:13.200923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229af38-3847-4c65-8365-090be7becbaf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:13.283831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:13.284749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:13.285032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:13.285314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:13.310218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:13.318992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:13.320807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:13.322454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 204/388] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:13.330901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:13.340196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee7ea-0228-47ee-91d5-957fce1c3a77 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:13.412928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:13.415073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:13.415922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:13.416687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:13.419565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:13.422838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:13.427095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 185/389] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:13 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:13.452208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:13.586257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:13.620937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578ec2-3bcb-4dc4-b63a-5ba676bc917c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:13.706059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:13.707922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:13.708671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:13.709477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:13.711475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:13.712626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:13.715302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:13.733133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 205/390] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:13.742533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:13.927663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77989951-0c18-44c6-b5da-2715b600ddc7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:13.930148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33833a9a-a7e8-49dc-95ba-07b01a7d6b08 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:13.942169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:13.977529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:13.979119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:13.979609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:13.980121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/groups/ea36716f35574d55bf51ca97fa6834fc/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:13.984364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, group_id=ea36716f35574d55bf51ca97fa6834fc, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:13.987873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876521b9-ce78-44c1-a2f0-94e35aef7c04 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:14.002666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:14.003734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.003969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.004278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.006912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:14.010415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d780a-2dc7-4d5f-81f5-918100adba29 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:14.014510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 206/391] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:13 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:14.017006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128f29-9115-4841-b117-1c3fe5256eab None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.017370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128f29-9115-4841-b117-1c3fe5256eab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.017706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128f29-9115-4841-b117-1c3fe5256eab None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.025419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 207/392] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:14.025419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6128f29-9115-4841-b117-1c3fe5256eab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:14.033852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 186/393] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 25 16:08:13 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/groups/ea36716f35574d55bf51ca97fa6834fc/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:08:14.039752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.040897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.042811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.045538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:14.090672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:14.095075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:14.276439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 187/394] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:14.317088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:14.337800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6128f29-9115-4841-b117-1c3fe5256eab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:14.392759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:14.394246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.394724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.395227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.396848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:14.398719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:14.401258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 208/395] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:14 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:14.410106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:14.521563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:14.526212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d8e0b-aacb-44ba-8013-032ecddcf606 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:14.600625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:14.602188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.602681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.603430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.627913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:14.640933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:14.652786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:14.653984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:14.654900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:14.656216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 188/396] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:14.663341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:14.740862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca846af-2963-464d-8e59-fb5f5e2fd23a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:14.878509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01850-f823-441e-a3ba-def43560a0a7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:14.914342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:14.915297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.915585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.915865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/groups/ea36716f35574d55bf51ca97fa6834fc/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.926403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, group_id=ea36716f35574d55bf51ca97fa6834fc, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:14.927947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:14.954460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:14.955852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:14.956075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 209/397] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 25 16:08:14 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/groups/ea36716f35574d55bf51ca97fa6834fc/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:08:14.956816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:14.957407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:14.959095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:14.959652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:14.962607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e41a8f-4ebb-4bc5-9b96-4851f8be2bc3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:14.981619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 189/398] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:14.987526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:15.010169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a4b5-acdf-4b49-9c0c-890792c4cb2b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.010739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a4b5-acdf-4b49-9c0c-890792c4cb2b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.011169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a4b5-acdf-4b49-9c0c-890792c4cb2b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.013281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 190/399] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:15.019500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.020093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.020565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.064878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:15.067995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:15.117363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:15.186256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd429-88e1-4ea6-bf11-e9b44d5c78d3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:15.239410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:15.240760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.241235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.241753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.243407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:15.245190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:15.247988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 210/400] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:14 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:15.254311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:15.265908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:15.289918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 191/401] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:15.319491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:15.423155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe96123-7b6e-4ed6-ac52-d0d5fd89ff5a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:15.486487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:15.487596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.488599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.488886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.490759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:15.492831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:15.500786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 211/402] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:15 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:15.501856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:15.631736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e162f3-aaa4-456b-8585-147d53eba3ec None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:15.676825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:15.747565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:15.749449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.752656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.753465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.764390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abaad2-bb98-4d6f-93d8-458e479efb30 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:15.801373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:15.821682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:15.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:15.824867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:15.830404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 192/403] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:15.839150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:15.853316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:15.854303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:15.854584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:15.854867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/groups/ea36716f35574d55bf51ca97fa6834fc/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:15.857898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c, group_id=ea36716f35574d55bf51ca97fa6834fc, role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:15.859551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0c57-2257-4a16-b8b4-f11403d454fa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:15.878795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 212/404] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 25 16:08:15 2022] PUT /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c/groups/ea36716f35574d55bf51ca97fa6834fc/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:15.888148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:16.103877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d622a3-a5d7-466c-a51b-67312c66c32b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:16.138471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b0238-eeca-4306-8785-41d20f2c9044 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.139381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b0238-eeca-4306-8785-41d20f2c9044 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.140013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b0238-eeca-4306-8785-41d20f2c9044 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.143776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 213/405] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:16.154509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.155231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.155921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.192970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:16.193879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:16.194074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.194541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.195251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.195948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:16.196628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:16.197154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:16.199747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:16.216017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 193/406] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:16.225124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:16.343319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 214/407] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:16.367985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:16.387337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:16.434188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130c881-ca92-4b31-9ce0-32adef06351f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:16.513395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:16.515046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.515547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.516083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.518556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:16.521958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:16.526671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 194/408] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:16 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:16.536177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:16.549063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:16.637910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd677d91-f709-47e8-b4f9-57446ffaeb3d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:16.721575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:16.725184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.725825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.726374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.773927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:16.800434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:16.802228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:16.803796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:16.805998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 215/409] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:08:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:16.817610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:16.854432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8753df-e18b-4ecc-841d-6e4234567251 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:16.971662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:16.974089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:16.974089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:16.975470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:16.978785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:16.981759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:16.987103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 195/410] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:16 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:16.997801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:17.029939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:17.191289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae6172-1fa0-45bb-9899-567228a81bc0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:17.265061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:17.300856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b3a72-265a-4461-bb62-a7b76c3396fe None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:17.308253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:17.309422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.309932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.310427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.312321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:17.312868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:17.315721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:17.333000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 216/411] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:17.341866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:17.442325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:17.443249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.443564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.443767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad/groups/ea36716f35574d55bf51ca97fa6834fc/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.458924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae85123e50c4564bc6f70df335004ad, group_id=ea36716f35574d55bf51ca97fa6834fc, role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:17.461014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:17.505273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 196/412] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 25 16:08:16 2022] PUT /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad/groups/ea36716f35574d55bf51ca97fa6834fc/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:08:17.541754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b4377-13e6-40d3-9794-ed232b1aba81 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:17.586373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:17.587409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.587551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.587914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] PATH_INFO: `/v3/groups/8b12503dafb14f098ed7b9ac60a43129` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.589074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] RBAC: Authorizing `identity:get_group(group_id=8b12503dafb14f098ed7b9ac60a43129)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:17.590698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6fbff-50ec-4e6a-abe4-00a32d27a6e8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:17.592871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 217/413] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:17 2022] GET /identity/v3/groups/8b12503dafb14f098ed7b9ac60a43129 => generated 229 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:17.599153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:17.766094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:17.850255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac5bf-fe3e-4df2-abe7-92a5e37eb792 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:17.870108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da38a8-ca35-4380-8ef7-1b3d079c700d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.870570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da38a8-ca35-4380-8ef7-1b3d079c700d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.870752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da38a8-ca35-4380-8ef7-1b3d079c700d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.872885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 218/414] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:17.881939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980c93-2414-4211-8f5e-178108ced77f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.882779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980c93-2414-4211-8f5e-178108ced77f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.883272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980c93-2414-4211-8f5e-178108ced77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.918068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-82980c93-2414-4211-8f5e-178108ced77f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:17.919963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82980c93-2414-4211-8f5e-178108ced77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:17.932832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:17.932832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:17.932832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:17.932832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] PATH_INFO: `/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:17.936627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] RBAC: Authorizing `identity:get_project(project_id=b8ee5c3834cf43d4a8d8dd7aee49ae24)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:17.940431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:17.944563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 197/415] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:17 2022] GET /identity/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24 => generated 344 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:17.955969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:18.143281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd434e-e7e6-4a03-9a51-5233ebe05306 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:18.169772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:18.187994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:18.188934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:18.189248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:18.189549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] PATH_INFO: `/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24/groups/8b12503dafb14f098ed7b9ac60a43129/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:18.192054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8ee5c3834cf43d4a8d8dd7aee49ae24, group_id=8b12503dafb14f098ed7b9ac60a43129, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:18.193831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:18.198076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:18.216923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 198/416] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 25 16:08:17 2022] PUT /identity/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24/groups/8b12503dafb14f098ed7b9ac60a43129/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:18.228661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 219/417] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:18.307077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:18.425179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04ea6-c85b-48ec-8c1d-dec645ca4ad4 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:18.425620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04ea6-c85b-48ec-8c1d-dec645ca4ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:18.425847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04ea6-c85b-48ec-8c1d-dec645ca4ad4 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:18.428620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 220/418] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:18.438427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:18.438427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:18.438427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:18.466069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:18.468108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:18.492009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd4f9-5701-4d9d-9d09-f624dbc9a9c0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:18.578627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:18.580012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:18.580525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:18.580525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:18.596821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:18.599598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 221/419] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:18.615546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:18.617412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:18.619267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 199/420] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:18.628086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:18.629218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:18.737993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:18.850023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74830d5-1c4a-4fae-9632-6f7cf0d965f5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:18.922548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7cddf-45ce-401b-b625-f0654d18349d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:18.951605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:18.962905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:18.964710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:18.965208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:18.965869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.005167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:19.024580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:19.026404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.027215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.028027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.029953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:19.034385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:19.037685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:19.040263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:19.042120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 16:08:19.043735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:19.053279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 200/421] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:19.069413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 222/422] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:19.072063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:19.083872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:19.301542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a9baa-6fb4-4cd5-8e72-e48ab5c2643f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:19.371370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:19.372591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.373144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.373680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.449888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a1f2c-dc5a-4702-9300-ea3e210d3cf3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:19.493984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:19.530625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:19.534739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:19.536883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.537427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.537933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.540187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:19.540747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:19.543650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2fce5-458c-48ff-bba9-15055b45d72b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:19.564537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 201/423] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:08:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:19.567281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb38f-9c7c-4a4f-9eee-4d748fe91f05 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.567766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb38f-9c7c-4a4f-9eee-4d748fe91f05 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.568248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb38f-9c7c-4a4f-9eee-4d748fe91f05 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.572174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 202/424] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:19.573736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cb38f-9c7c-4a4f-9eee-4d748fe91f05 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:19.575039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:19.576832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f24f4-951d-469c-94bc-a77378546eed None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:19.769807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cb38f-9c7c-4a4f-9eee-4d748fe91f05 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:19.789443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 223/425] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:19.792058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.792905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.793195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.820654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:19.821736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:19.822194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:19.822659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:19.826496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:19.828894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:19.890392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:19.923839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:19.926260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:19.962763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 224/426] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:19.964559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:19.980655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:20.078476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 203/427] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:20.079851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.176970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5c83-2f8d-4f44-99c3-f5cfc77c7a7b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.241767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:20.250801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560c630-b107-4892-b498-0986664ca433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.252726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c630-b107-4892-b498-0986664ca433 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.253506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c630-b107-4892-b498-0986664ca433 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.254135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c630-b107-4892-b498-0986664ca433 None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:20.256879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560c630-b107-4892-b498-0986664ca433 None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:20.260328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560c630-b107-4892-b498-0986664ca433 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:20.264251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 225/428] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:19 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:20.266150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560c630-b107-4892-b498-0986664ca433 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.298988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d32c-8bb4-4ce8-b322-494f6e2273c5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.378078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.379776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.380446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.381101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:20.401678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:20.416072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:20.417864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:20.419745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:20.421627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:20.426280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 204/429] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:20.428161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.472557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560c630-b107-4892-b498-0986664ca433 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.542289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.544266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.545044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.545805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:20.548330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:20.551339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:20.555527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 226/430] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:20.557516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.664783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44961e-b779-41fd-aee9-354c0af79a14 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.737655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.740050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.740843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.741660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:20.744285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:20.747363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:20.757567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 205/431] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:20.759497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.764963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b49f-aa1a-4ada-b149-11c58e2914d8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.780740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:20.840444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.842756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.843551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.844291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:20.846092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:20.846887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:20.849880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:20.866886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 227/432] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:20.868203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:20.877297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:20.950358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eb381-60ad-4f9b-8fad-d1da16147401 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:20.998362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:20.999665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:20.999665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:20.999953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.001408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:21.002926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94c602-94d8-4728-bf5b-5296b836294b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 206/433] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7935-dcc9-4431-8d30-559093a218ef None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7935-dcc9-4431-8d30-559093a218ef None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7935-dcc9-4431-8d30-559093a218ef None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 207/434] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:21.009515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7935-dcc9-4431-8d30-559093a218ef None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:21.080861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3600af-d62a-4158-99af-5b40df21074c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:21.122409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:21.148806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:21.149599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.149922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.150213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.151081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:21.151495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:21.153059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:21.161151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 228/435] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:20 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 611 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:21.162278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:21.190080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7935-dcc9-4431-8d30-559093a218ef None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:21.272425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:21.284079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:21.287297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.288309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.290217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.350326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ede2b-3283-4822-8170-23399143764f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:21.401566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:21.403209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:21.408717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:21.409652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.410058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.410481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.411503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=c618a32897ba4eb08e80389e7b3d1cc9, scope.project.id=6ae85123e50c4564bc6f70df335004ad) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:21.411957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:21.414131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9dd8e-bd47-48db-b600-cae101032ad4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:21.423612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 229/436] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:21 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=c618a32897ba4eb08e80389e7b3d1cc9 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:21.426390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236a397-6b6f-45bc-9c94-c21f4f9347dc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.426390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236a397-6b6f-45bc-9c94-c21f4f9347dc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.426390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236a397-6b6f-45bc-9c94-c21f4f9347dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.461302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3236a397-6b6f-45bc-9c94-c21f4f9347dc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:21.463677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3236a397-6b6f-45bc-9c94-c21f4f9347dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:21.549908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:21.602171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 230/437] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:21.604013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664dc-d9b1-4107-b31f-755c844e6769 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.604299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664dc-d9b1-4107-b31f-755c844e6769 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.604584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664dc-d9b1-4107-b31f-755c844e6769 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.606584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 231/438] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:21.613039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 208/439] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:21.616764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.617616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.618310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.619569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:21.648184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:21.650198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:21.776233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:21.798660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 232/440] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:21.800412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:21.803838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b5838-e4ee-45e8-a323-a55b7aff42cb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:21.873295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:21.874285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:21.874612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:21.874892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] PATH_INFO: `/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:21.876275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] RBAC: Authorizing `identity:get_project(project_id=7b0f08c7b99b41ae837c5bd48f2c1a7c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:21.878250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:21.880287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 209/441] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:21 2022] GET /identity/v3/projects/7b0f08c7b99b41ae837c5bd48f2c1a7c => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:21.888874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:21.969257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:21.986274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44584-bcdd-4f26-80ba-2a1c2ebf26b7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.054035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.055347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.055838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.056316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.056500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0789c7-2d66-4417-b7ed-fe653c732f8b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.076887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:22.092478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:22.094075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:22.095267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:22.101565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 233/442] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:22.101930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.102692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.103264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.103264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.103827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.121702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:22.137176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:22.138263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:08:22.139177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:22.141638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 210/443] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:08:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:22.142939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.185896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.261995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd387b-d2af-45da-994f-cad1f8f5bf3e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.269501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a954-9931-4328-965d-d89406b2d384 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.271722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.318673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.319396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.319846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.320212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.321045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:22.321595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:22.323183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:22.328693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.329630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.329937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.330184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.331638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:22.333287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:22.335161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 234/444] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:22.343705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.343992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 211/445] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:22.344888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.477246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803286-e8f1-404a-ab6b-bf1c4a21831c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.498245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e7d8-e581-4e08-acde-0f9d01fe4bf8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.502787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.529881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.531039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.531521 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.531997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.533419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:22.533959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:22.536638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:22.543811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.544569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.544909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.545273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.561483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 235/446] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:22.561483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.602741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.657959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:22.659971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:22.707713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d135c-5dec-4ebe-882f-0d48f1f74d7f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:22.772809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:22.773680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:22.774047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:22.774524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:22.776308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:22.776998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:22.778729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:22.785596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 236/447] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=7b0f08c7b99b41ae837c5bd48f2c1a7c&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:22.786479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.832653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 212/448] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:22.834584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:22.887352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.935210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:22.994472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b687e-eb45-44a8-8d84-58e3c1d55958 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.003856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4255f-d78a-4407-a099-326a1c3eb42c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.057315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.058532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.059023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.059546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.063474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.065238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.065758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.066253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] PATH_INFO: `/v3/projects/6ae85123e50c4564bc6f70df335004ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.070538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae85123e50c4564bc6f70df335004ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.072344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.079814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 213/449] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/projects/6ae85123e50c4564bc6f70df335004ad => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:23.082337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:23.198608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.201086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.267163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df274-e0c1-48e0-a3b5-6baf9ca91497 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.285666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:23.316539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:23.325322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.326557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.326839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.327276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] PATH_INFO: `/v3/groups/ea36716f35574d55bf51ca97fa6834fc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.329757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] RBAC: Authorizing `identity:get_group(group_id=ea36716f35574d55bf51ca97fa6834fc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.332572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.336452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 214/450] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:23 2022] GET /identity/v3/groups/ea36716f35574d55bf51ca97fa6834fc => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:23.348771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:23.403586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 237/451] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:23.411861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:23.542098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe32cc-3748-47e8-9b9c-25e566e344e4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.583823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.584501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.584780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.585053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.585976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] RBAC: Adding query filter params (group.id=ea36716f35574d55bf51ca97fa6834fc, role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=6ae85123e50c4564bc6f70df335004ad) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:23.586284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.587798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.591543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a381c30-1460-42a6-be54-ae6d2c40731f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.594744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 215/452] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:23 2022] GET /identity/v3/role_assignments?group.id=ea36716f35574d55bf51ca97fa6834fc&scope.project.id=6ae85123e50c4564bc6f70df335004ad&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:23.663451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.665020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.665542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.666010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] PATH_INFO: `/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.668462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] RBAC: Authorizing `identity:get_project(project_id=b8ee5c3834cf43d4a8d8dd7aee49ae24)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.671453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.674931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 238/453] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:08:23 2022] GET /identity/v3/projects/b8ee5c3834cf43d4a8d8dd7aee49ae24 => generated 344 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:23.684605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:23.772538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:23.829978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a4a79-0c18-4d65-80e2-8c06c9eb84a9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:23.880072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:23.880995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:23.881258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:23.881701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] PATH_INFO: `/v3/groups/8b12503dafb14f098ed7b9ac60a43129` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:23.883429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] RBAC: Authorizing `identity:get_group(group_id=8b12503dafb14f098ed7b9ac60a43129)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:23.884932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fcb53-835f-4053-b636-f23e8d93859f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:23.886777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 216/454] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 25 16:08:23 2022] GET /identity/v3/groups/8b12503dafb14f098ed7b9ac60a43129 => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:23.893633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:23.914070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:24.038829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0f5d-3f0b-441e-8241-4317cd87adee None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:24.102548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:24.103623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:24.104116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:24.104615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:24.106324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] RBAC: Adding query filter params (group.id=8b12503dafb14f098ed7b9ac60a43129, role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=b8ee5c3834cf43d4a8d8dd7aee49ae24) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:24.106887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:24.109693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003a016-3e9b-4bb2-91ef-1b90811285ae None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:24.119925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 239/455] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 25 16:08:23 2022] GET /identity/v3/role_assignments?group.id=8b12503dafb14f098ed7b9ac60a43129&scope.project.id=b8ee5c3834cf43d4a8d8dd7aee49ae24&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:28.112533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b61-48c2-4bb8-8121-2bdcd5874b24 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.113367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b61-48c2-4bb8-8121-2bdcd5874b24 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.113800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b61-48c2-4bb8-8121-2bdcd5874b24 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.116841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 217/456] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:28.126913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755afd8-359d-4912-99eb-8697ce31ba36 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.127702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755afd8-359d-4912-99eb-8697ce31ba36 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.128028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755afd8-359d-4912-99eb-8697ce31ba36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.175303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c755afd8-359d-4912-99eb-8697ce31ba36 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:28.177431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c755afd8-359d-4912-99eb-8697ce31ba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:28.350044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 240/457] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:28.419209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38b61-48c2-4bb8-8121-2bdcd5874b24 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:28.523621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454345d4-80ec-42ba-a988-ee2fe09cd1c2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.525753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454345d4-80ec-42ba-a988-ee2fe09cd1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.529299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454345d4-80ec-42ba-a988-ee2fe09cd1c2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.532475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 241/458] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:28.553222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803884f-415a-4a85-b293-76cab16da65f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.554306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803884f-415a-4a85-b293-76cab16da65f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.555088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803884f-415a-4a85-b293-76cab16da65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.619110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4803884f-415a-4a85-b293-76cab16da65f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:28.621062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4803884f-415a-4a85-b293-76cab16da65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:28.624856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:28.686974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38b61-48c2-4bb8-8121-2bdcd5874b24 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:28.754644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:28.766232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:28.767739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.768903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.769426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.795019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:28.798427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:28.802161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 242/459] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:28.805240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaed08-fa2f-43f9-8e3b-8659eb44f22f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.806011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaed08-fa2f-43f9-8e3b-8659eb44f22f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.806695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaed08-fa2f-43f9-8e3b-8659eb44f22f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.815522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 243/460] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:28.820909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:28.821764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:28.822449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:28.837092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:28.839067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:28.840816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:28.840816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:28.840816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:28.840816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:28.840816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 218/461] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:28.851497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:28.864401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:28.867268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:29.010870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 244/462] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:29.012014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.026504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e235f-889b-4fb8-9ff2-b92a6b932c98 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.095538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:29.097272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.098268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.099073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.100836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:29.101864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:29.105075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:29.114798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:29.131511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 219/463] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:29.139339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.169934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f73e-292a-41ae-923e-a98227645d08 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.234863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:29.244201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:29.246326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.247068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.247703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.270748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:29.274304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:29.314045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:29.315854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:29.317236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:29.317236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:29.317236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:29.317236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:29.320216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 245/464] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:29.321995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.351392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a469a-30db-48ee-b923-5bd7da441517 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.420923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:29.423090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.424073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.424877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.451547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:29.454966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:29.486549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:29.487537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:29.488444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:29.488444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:29.488444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:29.488444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:29.491317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 220/465] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:29.492520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.542189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4dd-af37-4fb1-bdf2-ac9d4a429937 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.582846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:29.616439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:29.617667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.618159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.618637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.619998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:29.620430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:29.623210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:29.627808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a380c6-fe68-49ac-94fe-b4dcb551186b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.669375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 246/466] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:08:29 2022] POST /identity/v3/users => generated 246 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:29.669375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.696104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:29.697315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.698039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.698683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.700162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:29.700835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:29.703714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c27788-ca94-4fe8-af0e-1d0672d49bbb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:29.728030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 221/467] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:29.730892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7955-ca85-47bc-93b1-b4643597f3e7 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:29.731611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7955-ca85-47bc-93b1-b4643597f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:29.732211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7955-ca85-47bc-93b1-b4643597f3e7 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:29.735292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 222/468] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:29.743073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7955-ca85-47bc-93b1-b4643597f3e7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:29.920656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a2a9-6866-496f-8cc8-8d82df78cf67 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:29.960956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7955-ca85-47bc-93b1-b4643597f3e7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:30.000977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:30.004645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.004645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.004645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.004645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:30.005274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:30.008261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77b6e-1eae-4cfa-b86b-1c3efa11d0d5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:30.041201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:30.042398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.042889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.043371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.044725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:30.045154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 247/469] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:30.047460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:30.047682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8f832-0aff-4190-82ab-2fef26bfbd36 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.048008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8f832-0aff-4190-82ab-2fef26bfbd36 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.048484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8f832-0aff-4190-82ab-2fef26bfbd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.083290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-17a8f832-0aff-4190-82ab-2fef26bfbd36 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:30.086175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-17a8f832-0aff-4190-82ab-2fef26bfbd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:30.089081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 223/470] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:08:29 2022] POST /identity/v3/users => generated 248 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:30.090366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:30.137831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:30.150637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:30.251078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 248/471] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:30.254873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46f3d-2f48-4931-86ce-ccbc66de648a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.255730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46f3d-2f48-4931-86ce-ccbc66de648a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.256415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46f3d-2f48-4931-86ce-ccbc66de648a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.263906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 249/472] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:30.275937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.276799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.277595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.332744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:30.334650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3f8b-d89f-4d1e-9fc3-cd8b84494256 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:30.334898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:30.404147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:30.404864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.405155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.406102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.407048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:30.408690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:30.445093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 224/473] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:08:30 2022] POST /identity/v3/users => generated 248 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:30.446169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:30.526744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 250/474] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:30.562154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:30.565917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:30.618095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629553e-9e11-44e9-942a-b5030ae31c37 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:30.686579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:30.687982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.688455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.688933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.744827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:30.747706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:30.778481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:30.780053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:30.781663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:30.781663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:30.781663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:30.781663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:30.785567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 225/475] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:30.787631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:30.806040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:30.812781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7bd9-4e82-4a4e-903f-05f347e17aed None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:30.888700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:30.889601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:30.889896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:30.890730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:30.891739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:30.893881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:30.909525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.sql.core [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 16:08:30.909525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 16:08:30.909525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [parameters: {'id': 'd9418583d07a4070b3b83da3efddf23b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 16:08:30.909525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80225) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 25 16:08:30.948597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:08:30.950364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application yield resource Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise self.value Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application session.commit() Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 16:08:30.952976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.flush() Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application compat.raise_( Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 16:08:30.956037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:08:30.966314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:08:30.968261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [parameters: {'id': 'd9418583d07a4070b3b83da3efddf23b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:30.970277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 16:08:30.972158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:30.973705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 251/476] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 25 16:08:30 2022] POST /identity/v3/roles => generated 144 bytes in 386 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:30.973828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff86-b275-4223-b14f-12862b57fd8b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:30.973948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:31.000069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:31.035001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:31.036198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:31.036825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:31.037382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:31.039170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:31.039774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:31.044984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:31.074862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 226/477] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:31.076286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:31.261388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc26eb-3b8e-4906-8392-f29bdedc77c8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:31.292610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670fc3c-6494-43cb-b260-2c308c9925c7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:31.370554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:31.371927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:31.372402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:31.372855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:31.391331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:31.392064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:31.392351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:31.392632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:31.393452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:31.394892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:31.395068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f1f7-0e2c-42a0-87dd-ac8df071e915 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:31.409447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:31.411164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:31.412600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 252/478] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:08:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:31.421086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:31.461375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 227/479] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:08:31 2022] POST /identity/v3/users => generated 249 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:31.478223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:31.602810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eb130-8dda-4c32-bb11-1ea1afe3551c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:31.678807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:31.679992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:31.680495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:31.680940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:31.682619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:31.683013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:31.684446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e3bb4-a3fb-4aca-8875-e9bc254c36b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:31.699873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 253/480] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:33.544507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4cd76-a343-4e5d-879f-46e68639edac None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:33.545631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4cd76-a343-4e5d-879f-46e68639edac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:33.546116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4cd76-a343-4e5d-879f-46e68639edac None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:33.547861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 228/481] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:08:33.554487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11248-22c7-4e67-9f63-563fd3e1703e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:33.555011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11248-22c7-4e67-9f63-563fd3e1703e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:33.555120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11248-22c7-4e67-9f63-563fd3e1703e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:33.583376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d9e11248-22c7-4e67-9f63-563fd3e1703e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:33.584818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d9e11248-22c7-4e67-9f63-563fd3e1703e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:33.738749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 254/482] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:33.766624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4cd76-a343-4e5d-879f-46e68639edac None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:33.816533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7459-4235-4475-970b-8f36fe2d7668 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:33.817054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7459-4235-4475-970b-8f36fe2d7668 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:33.817619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7459-4235-4475-970b-8f36fe2d7668 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:33.820546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:33.820846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 255/483] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:33.828917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:33.830417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:33.831064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:33.899439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:33.901720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:33.921710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4cd76-a343-4e5d-879f-46e68639edac None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:33.941966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:33.972551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:33.974066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:33.974350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:33.974652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:33.992567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:34.002236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:34.004000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:34.005584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 229/484] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:34.012104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.082113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 256/485] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:34.123104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.142335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:34.150985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63479ad-d306-4171-9e73-4f65bed49cfa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.205160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:34.206366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.206865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.207378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.208793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:34.209373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:34.212700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:34.234985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 230/486] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:34.241732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.404294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b0c00-e1a1-4690-a8dc-1574dd696bba None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.450313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce11c2-fb09-45c4-8bbd-48abe3ed12eb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.479500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:34.480870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.481429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.481867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.499060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2902b9b1-4827-488f-8bf2-c688e380b46d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:34.509835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:34.510888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:34.511770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:34.514716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 257/487] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:34.516689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325e3d-c012-47e0-9b00-64d1b057b6e8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.517177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325e3d-c012-47e0-9b00-64d1b057b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.517742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325e3d-c012-47e0-9b00-64d1b057b6e8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.523885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 258/488] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:34.523885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e325e3d-c012-47e0-9b00-64d1b057b6e8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.525144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:34.526653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.527217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.527748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.533576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:34.551287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:34.554047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c71d52bb-ba19-4008-990d-e09ca81a32c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:34.588434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:34.589371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:34.590243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:34.590243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:34.590243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:34.590243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:34.597531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 231/489] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:34.600252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.600741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.601231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.634386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:34.637372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:34.642368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:34.682880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e325e3d-c012-47e0-9b00-64d1b057b6e8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.743781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:34.745231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.745877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.746520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:34.748088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:34.748799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:34.751470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:34.770070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 232/490] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:34.772900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.781890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 259/491] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:34.792552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:34.887973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:34.947450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29106c-f7d1-4b14-94d5-69913a99acc8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.987532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12575c-4a59-464e-9529-f56326a2ec7a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:34.995149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:34.996035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:34.996320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:34.996598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.011185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.012873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:35.034023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:35.035270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:35.036392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:35.036392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:35.036392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:35.036392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:35.036392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 260/492] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:35.036795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.054899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.055601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.055874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.056151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.057289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:35.057879 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.060398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7dad3-6dbd-4569-b7bd-33526850f211 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.079211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:35.089652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 233/493] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:35.091713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad344a6-efdf-4128-bc96-0e23a626b783 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.092053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad344a6-efdf-4128-bc96-0e23a626b783 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.092294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad344a6-efdf-4128-bc96-0e23a626b783 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.094187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 234/494] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:35.095452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad344a6-efdf-4128-bc96-0e23a626b783 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.213565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402c36-bf47-4f3e-a756-6a5a1450093c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.256423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:35.270020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad344a6-efdf-4128-bc96-0e23a626b783 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.291397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.293041 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.293609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.294166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.316712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:35.329931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.330687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.330996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.331383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d6dcc767-dfb8-4dda-bd36-4fce189794dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:35.331858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:35.333823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:35.336594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:35.339274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:35.339274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.339657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 261/495] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:35.339825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080c527-1418-4ed4-884b-7094c4272d71 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.339995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a043-0f3b-4a96-9be5-c36590c39a70 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.339995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a043-0f3b-4a96-9be5-c36590c39a70 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.339995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a043-0f3b-4a96-9be5-c36590c39a70 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.340627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 262/496] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:35.345277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6a043-0f3b-4a96-9be5-c36590c39a70 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.352392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 235/497] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:35.353981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.354293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.354548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.375001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:35.376266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:35.448702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:35.464003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 236/498] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:35.466894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.521924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6a043-0f3b-4a96-9be5-c36590c39a70 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.574064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.574980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.575265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.575537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.585484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.587033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a3f58bd4-44de-4412-b25e-a28e03d06e1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:35.614777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:35.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:35.618495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 263/499] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:35.619602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.619956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.620795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.628268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2522c-e52d-46b9-a071-7583dbefb65f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.652838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:35.655055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:35.673101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.673864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.674148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.674429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.675219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:35.675525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.676980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.690311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 237/500] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:35.692098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.711705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:35.765954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:35.781239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 264/501] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:35.782211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.836650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84028302-8c12-41d5-b7d7-a0c4b61209c9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.889305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.890161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.890442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:35.900678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:35.902475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:35.920529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:35.921713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 238/502] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:35.923250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:35.942486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d1dd5-070e-4305-a45f-9eb6d4f65a12 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:35.987570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:35.988416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:35.988704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:35.988973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.002218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:36.010769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:36.012150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 265/503] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:36.013635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.060477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:36.106705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7fc3-767a-4732-9233-1c4d3b4e54c5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.128294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:36.169892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.171095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.171592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.172101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.173949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:36.174504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:36.177396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:36.190769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ba28-b728-47a9-af18-4aca576ede90 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.245611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.246932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 239/504] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:08:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:36.247122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.247485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.247809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.248318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.265781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:36.268006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:36.288743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:36.290529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 266/505] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:36.292196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.416082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:36.440406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc036f-bb1e-439d-9349-c7ce04393ffb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.456311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee9fda-1979-42a5-a4a9-4da634c468c0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.514951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.516412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.516985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.517521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.528243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.529481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.529978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.530483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.532227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:36.532722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:36.535627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:36.538541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:36.540555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:36.569101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:36.570475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:36.571405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:36.571405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:36.571405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:36.571405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:36.571617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 240/506] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:36.573193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.599164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 267/507] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:36.600359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.627488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:36.789151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be6ea-276c-4d90-a3ec-ca44a588a3e7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.801834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d470-0d63-41db-804d-24b99ebab8f9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:36.853099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.853848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.854190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.854512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.855339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:36.855648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:36.857462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:36.867960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 241/508] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:36.868896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:36.874384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:36.875583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:36.876087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:36.876585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:36.884062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.007546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.010257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.020580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.049178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36369dc6-4cc5-4f00-a46a-68ac51340887 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.111496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.112683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.113192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.113810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.115176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:37.115683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.118462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.138770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 242/509] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:37.139684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.195603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 268/510] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:37.196506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.259660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.267382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87e81b-e4a8-46ec-a693-b8816988b336 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.322411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.323178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.323564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.323965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.324886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:37.325324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.327010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.341096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 243/511] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:37.341878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.350409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.355870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a09f-65f7-4000-8cd5-96b7b27f724a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.404723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.405419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.405768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.406046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.495712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636fd06-aa66-454e-9b0c-c3e0d21bd434 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.510015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.511978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.542634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.543850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.544271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.544734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.555844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.560008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.561717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:37.583113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:37.585081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 244/512] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:37.586835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.663004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 269/513] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:37.664687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.667043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.745256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9f56-350c-4128-bbc9-0935b83cc30e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.790228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.791107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.791402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.791677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.802251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.803988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:37.822124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:37.823973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 245/514] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:37.826409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:37.854375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe083f-5a89-4f2a-b39b-3ca2cfd55737 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:37.910808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:37.911697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:37.912023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:37.912306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:37.916392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:37.927260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:37.928990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:37.949501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:37.951142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:37.952803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:37.952803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:37.952803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:37.952803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:37.952803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 270/515] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:37.953292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.018267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd13df-58f5-426b-9f02-80139a8e7e78 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.039977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:38.067569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.068372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.068738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.069137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.070112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.071738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.102081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 246/516] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:08:37 2022] POST /identity/v3/users => generated 247 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:08:38.103673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.133024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cef8d-494b-46fd-86c1-88edbe10c5f4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.207246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.208522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.209009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.209552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.211077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:38.211653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.213942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.246107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 271/517] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:38.246107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.346943 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb60b-21bd-45e2-8ee4-3ef67368d73d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.354780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:38.414003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.415821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.416648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.417427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.435986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.437750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:38.473970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:38.475218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:38.476774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:38.476774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:38.476774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:38.476774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:38.476774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 247/518] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:38.478280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.480078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92dc64-cf7b-4023-ae86-95f1409af8c4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.535238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:38.553967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.554757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.555040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.555318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.556700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:38.557225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.559162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.599282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 272/519] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:38.599282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.681938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe928a-f914-4f86-bb86-b39b40e21c55 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.745518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.746607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.747088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.747565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.748918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:38.749489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.751961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.775974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 248/520] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:38.775974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.782803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6245-6f93-4000-9818-d69cb26a3d7e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:38.795890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:38.845398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:38.846520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:38.846989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:38.847443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:38.868359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:38.870078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:38.896333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:38.898276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 273/521] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:38.901451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:38.957745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:38.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8324-289a-40eb-a271-3c1a01bc9dda None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.061619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.063439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.063809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.064085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.064884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:39.065315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.067584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.091091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 249/522] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:39.091091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:39.117307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428904-324c-4bcd-9e05-c7a1938920c1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.186784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.187999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.188510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.189211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.260140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:39.267218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89578f52-6d35-4746-b2f4-b7eb80901d77 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.330724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.331982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.332388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.333722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.334149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.334583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.347487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.349914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:39.371328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:39.372356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:39.373539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 250/523] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:39.374302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:39.380459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:39.530369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 274/524] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:39.531962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:39.558891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebdb1b-e676-41bb-9966-6466b9847dd4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.625968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.627185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.627686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.628182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.629923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:39.630477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.633737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.677423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 251/525] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:39.677423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:39.686903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:39.733715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d313-cff6-4137-88b7-90ce981ed65a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.785680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.786570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.786860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.787259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.792283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:39.797872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.800646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:39.829230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:39.830515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:39.831359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:39.831773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 275/526] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:39.833329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:39.880397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d21e6-98ff-4866-ad06-8446a894de41 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:39.933594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:39.934954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:39.935456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:39.935941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:39.937500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:39.937989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:39.940626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:39.978553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 252/527] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:39.980098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.033105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d099b-5383-47e9-9f16-bd96c8fb519c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.091542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.111593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.113018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.113711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.114190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.139159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.142140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.158406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585b718-10b0-4072-bdf1-048fd94817eb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:40.177711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:40.178737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:40.179583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:40.179583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:40.179583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:40.179583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:40.179802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 276/528] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:40.180977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.192508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.213550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.214262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.214789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.215252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.315095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.316628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.377073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711998f-fd79-4a0b-bf05-6137097bf5b9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.424507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.425581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.425946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.426515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.427619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:40.429198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.432154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.463808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 277/529] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:40.472682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.475372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 253/530] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:40.476538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.513276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.529553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.607963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6208-ca65-4a08-b478-2c2626376288 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.622714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72877916-ba01-44a4-a6e2-96dc649d1b15 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.658028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.658915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.659183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.659460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.660927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:40.661248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.663283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.666479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.667250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.667534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.667846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.668890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:40.669287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.670610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 278/531] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:40 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:40.671159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.682198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.709379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 254/532] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:40.710840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:40.885987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.896194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6f87-eca2-49d3-adb5-9a3c0ab77829 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.911731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:40.929941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980364a-6f30-4bd1-ad42-f0585c633401 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:40.977068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.979176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.980076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.980531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:40.981590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=0c159db46d48429b95e0166596c53d81) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:40.981999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:40.984152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:40.984951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:40.986562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:40.987188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:40.987957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.001457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 279/533] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:40 2022] GET /identity/v3/role_assignments?user.id=0c159db46d48429b95e0166596c53d81&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:41.002833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:41.034802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.036452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:41.071641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:41.072663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:41.073592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:41.073592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:41.073592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:41.073592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:41.077508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 255/534] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:41.079635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:41.260745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153699-8f8a-4a17-b58d-4c080035c92d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:41.333566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:41.335077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.335581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.336071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.350054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.353059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.367610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045110e9-f467-4384-8ae3-4e469994829c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6e8d878b-cc01-4995-b924-8ff638e94325 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:41.378945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:41.379970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:41.380930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:41.382490 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 280/535] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:41.389988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0657-e4f1-4d67-b276-b4d5a75b3128 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.390467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0657-e4f1-4d67-b276-b4d5a75b3128 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.390897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0657-e4f1-4d67-b276-b4d5a75b3128 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.393888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 281/536] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:41.395910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0657-e4f1-4d67-b276-b4d5a75b3128 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:41.438023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:41.438918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.439390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.439829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.440773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:41.441288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.443521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca38a4f-8001-4dbc-a0dc-e41f32f2ca67 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.445525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:41.447330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:41.467977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 256/537] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:41.467977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.468346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.468839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.496284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:41.498006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:41.571340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0657-e4f1-4d67-b276-b4d5a75b3128 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:41.609787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 257/538] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:41.611672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:41.635940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:41.637485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.638163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.638753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.640639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:41.641517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.644360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.660150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 282/539] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:41.670275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:41.846805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dafec-54a6-4c1d-bac0-c19c28f529ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:41.885846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:41.910876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708325f-eeb8-434c-b7e1-3932162c72b0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:41.914561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:41.922622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:41.924048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.924541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.925035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.941061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.944885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.981879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:41.982992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:41.983456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:41.983901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:41.984658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:41.986052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:41.986915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:41.988474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=468fc86f361a4c7c91cacc1d90086176) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:41.989078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:41.991045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:41.993850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 258/540] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:41.996370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:42.002936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 283/541] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/role_assignments?user.id=468fc86f361a4c7c91cacc1d90086176&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:42.013388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:42.328229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213280-12ca-4c93-8d65-d3b9c9aede95 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:42.333028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd182d-ed55-4ccf-8c2e-0f821eff2879 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:42.394743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:42.395603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:42.395974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:42.396254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:42.403656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:42.404829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:42.405313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:42.405886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:42.407437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:42.407976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:42.410892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:42.412760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:42.429611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:42.431374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:42.432944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:42.438199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 259/542] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:42.438955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 284/543] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:42.447016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:42.452686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:42.501832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:42.631652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:42.761794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741b8-a870-4f44-b785-b6872436f341 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:42.800734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba594-285a-4e65-ad32-700b4f0fb3de None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:42.838931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:42.840078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:42.840562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:42.841057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:42.842718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=8ff3caa305504fcd888458b91bf1fc5b) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:42.843293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:42.848180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:42.877577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 285/544] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:42 2022] GET /identity/v3/role_assignments?user.id=8ff3caa305504fcd888458b91bf1fc5b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:42.915536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:42.916320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:42.916625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:42.916952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:42.917823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:42.918137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:42.919730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:42.940643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 260/545] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:42.950147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:43.128175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:43.197613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b7f41-0fc8-4610-b290-aac162b593ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:43.287345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:43.288792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:43.289299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:43.289850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:43.315787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:43.318780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:43.367727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:43.370037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:43.371713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:43.371713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:43.371713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:43.371713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:43.371713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 286/546] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:43.379868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:43.586980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35dffd-144b-48cd-bac9-3a4b3dcb60c3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:43.596402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:43.645203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:43.646190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:43.647316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:43.648031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:43.649132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:43.649696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:43.651590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f32858-f03d-44c5-8475-d6ca7eba963b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:43.669680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 261/547] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:43.678652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:43.745695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc5fb-1a13-453b-8191-621d3960a5c6 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:43.745966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc5fb-1a13-453b-8191-621d3960a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:43.746302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cc5fb-1a13-453b-8191-621d3960a5c6 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:43.748707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 262/548] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:43.759855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:43.760363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:43.760812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:43.826615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:43.828431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:43.887952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd980f-0a91-473e-8d3c-3f492602c8b4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:43.925286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:43.946268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:43.947542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:43.948265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:43.948877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:43.968705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:43.972564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:44.011632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:44.013216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:08:44.014503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:44.017284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 287/549] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:08:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:44.035376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:44.037320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 263/550] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:44.089999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:44.283832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16352-12b9-4f5c-bd25-94b7ce0dcc66 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:44.380544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:44.391131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f2d60-dc9e-4623-9cc6-f07bc40d73b2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:44.391870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:44.393264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:44.393947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:44.394862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:44.396949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:44.397727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:44.400910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dd23f-1420-4633-affa-a542d4024367 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:44.443864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 288/551] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:08:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:44.446721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a8040-573b-4631-90e3-9e4174d09699 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:44.447509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a8040-573b-4631-90e3-9e4174d09699 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:44.448177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a8040-573b-4631-90e3-9e4174d09699 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:44.457644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 289/552] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:44.459722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a8040-573b-4631-90e3-9e4174d09699 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:44.483935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:44.486182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:44.487018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:44.488723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:44.511904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3747aae8-d0b0-40ac-8633-af3ea0d9b99a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:44.527122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:44.528210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:44.529143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:44.530182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 264/553] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:44.533455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:44.533455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:44.533749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:44.543087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:44.570737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:44.573505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:44.673030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a8040-573b-4631-90e3-9e4174d09699 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:44.728004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 265/554] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:44.730005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:44.769733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:44.771091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:44.771682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:44.772463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:44.941177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:44.967450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:44.970754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:44.979417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdde37c-2ccf-4300-af28-9f2c2fac6eca None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.052491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:45.053677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.054150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.054617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.056258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:45.056806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:45.059502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:45.076031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:45.079594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 266/555] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:45.082368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:45.218083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 290/556] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:45.219534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:45.283128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7322-cdbf-4905-afc7-613d594a7897 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.385238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:45.386693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.387263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.387526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.409638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:45.424857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:45.426795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:45.428388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:45.430911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af08c3-5ea6-46bc-87b2-1ff85e1725b6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.433299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 267/557] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:45.435079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:45.455435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:45.506600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:45.507716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.508112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.508408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.521700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:45.523542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:45.527818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4b6f8488-648f-4c3e-b225-68e3fc8bfbfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:45.553834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:45.556328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:45.560487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:45.560487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:45.560487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:45.560487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:45.562098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 291/558] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:45.563106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd703ba-0321-45cc-96e7-d0169b4f04ec None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.563906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd703ba-0321-45cc-96e7-d0169b4f04ec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.563906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd703ba-0321-45cc-96e7-d0169b4f04ec None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.565818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 292/559] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:45.571826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd703ba-0321-45cc-96e7-d0169b4f04ec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:45.602461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31d20-82bc-4ee4-9382-79c663b1beaf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.675045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:45.676490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.676989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.678509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.701310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:45.704247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:45.738457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:45.739628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:45.740783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:45.740783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:45.740783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:45.740783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:45.741131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 268/560] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:45.742536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:45.789428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd703ba-0321-45cc-96e7-d0169b4f04ec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.856139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:45.856781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.857165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.857630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.858531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:45.858846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:45.860398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290bf38-19e9-4610-a054-1a9ec7b054c2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:45.868074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:45.872836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 293/561] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:45.875746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525eba-66ad-4892-8267-52606043c122 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:45.876251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525eba-66ad-4892-8267-52606043c122 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:45.876751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525eba-66ad-4892-8267-52606043c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:45.902769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-55525eba-66ad-4892-8267-52606043c122 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:45.904168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-55525eba-66ad-4892-8267-52606043c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:45.948425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b888520-a58d-492e-89e8-0761f2b4652e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:45.982365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:46.009373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 294/562] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:46.009373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525eba-66ad-4892-8267-52606043c122 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.011026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.011779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.012058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.012330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.017530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:46.018723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.021619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.039858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 269/563] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:46.041072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.189966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525eba-66ad-4892-8267-52606043c122 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.232303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e869b-7750-4b75-a97c-229a21a0ccf0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.255601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.257692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.258211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.258718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.259217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:46.260167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:46.260801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.263753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.276445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.277274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.277589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.278015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.281408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 295/564] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:46.282826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.294860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.296284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:46.319984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:46.321242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 270/565] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:46.322833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.373017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:46.466323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205b67-d326-4d31-a881-e6c411eb6d81 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.488015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435585b-8bbf-4976-9d89-353b1b669e74 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.535868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.537297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.537894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.538372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.542942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.544217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.544546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.544849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.554083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.556976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.559950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-46279c1e-6740-4372-b3cd-6f0ebe92479e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:46.569659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:46.571500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:46.573096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 296/566] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:46.576122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c53fd-aa4f-40d8-9e5f-8dc2c91a2cd3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.576450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c53fd-aa4f-40d8-9e5f-8dc2c91a2cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.576878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c53fd-aa4f-40d8-9e5f-8dc2c91a2cd3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.580723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 297/567] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:46.580723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c53fd-aa4f-40d8-9e5f-8dc2c91a2cd3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:46.584016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:46.585968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 271/568] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:46.588390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.634249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:46.736505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f7fc-5488-438c-b2eb-13f34b235379 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.746983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c53fd-aa4f-40d8-9e5f-8dc2c91a2cd3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:46.760178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:46.792723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.793953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.794468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.794948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.796343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:46.796866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.797857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:46.798707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.799015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.799263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.799450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.807836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:46.809598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:46.815578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 272/569] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:46.816345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9f23995f-b1bb-493d-b909-5284a1641690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:46.829601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:46.831344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 298/570] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:46.833091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:46.834367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:46.856225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:46.857545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:46.967480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 299/571] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:46.968906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:46.983821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.001664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fa42f-8c6d-4056-85ca-150695577cca None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.064742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.065559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.065848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.066123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.067010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:47.067329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.068878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.081712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 273/572] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:47.082728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.129122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.143236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640ebc6-c5c9-46cf-8b73-4754e2d72583 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.208396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.209506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.210103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.210435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.211825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:47.212318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.215096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.243981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 300/573] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:47.245473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.289762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af530-56b9-4336-b7f3-1b585f6d511b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.350315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.351622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.352121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.352615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.367549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.370139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:47.395131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:47.397213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 274/574] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:47.399409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.423480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.428478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e94674-5ef5-4f58-9b52-ea328faaf3e5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.487975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.489160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.490117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.490261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.491676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:47.492216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.495097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.518545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 301/575] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:47.520062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.549293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.574705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628050-f6c2-4603-805e-83e29242e25a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.626656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.627409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.627700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.627984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.643030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:47.651313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:47.653275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:47.655146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:47.659882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 275/576] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:47.659882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.678654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c911e-9d72-4f6f-8073-d1ca0bb1e34f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.728902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.729787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.730072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.730345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.747174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.749564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:47.773969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:47.775715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 302/577] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:47.778017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.793661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.839058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85272ced-66e8-4b19-80dc-06a8b9c84e94 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:47.880298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:47.894100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:47.895221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:47.895541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:47.895910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:47.912242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:47.914027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:47.937042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:47.939167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 276/578] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:47.941257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:47.952095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ef2f1-a73f-4122-817b-7fd2f8034cad None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.016023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.017115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.017608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.018071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.019678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:48.020173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.023067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.047507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 303/579] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:48.048763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.106134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b4849-3357-4731-974e-2600006fa87b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.153288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:48.175863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.176934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.177264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.177971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.179908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=4f36fd1f37394b49ac38568200578c32) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:48.180458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.183178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.205854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 277/580] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:08:47 2022] GET /identity/v3/role_assignments?user.id=4f36fd1f37394b49ac38568200578c32&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:48.205854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.245838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20372fd-d9c8-4c99-9a94-969c2c13dafb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.305112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:48.323988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.325215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.325791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.326310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.327987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:48.328356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.331067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.354914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 304/581] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:48.354914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.413646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae4ef-f93e-4781-b774-992461c7c225 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.493822 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.494783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.495185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.495468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.496351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:48.496749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.498279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.529172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 278/582] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:48.529172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.550709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81185b7-86f3-4f9c-842d-86a82d2159b6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.607442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.608853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.609569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.610188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.611732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:48.612409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.615241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.636331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:48.638106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 305/583] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:48.639711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.742441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:48.753816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874ab6-baeb-452e-8088-4cb406d4d2cb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.815959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.816945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.817388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.817711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.833068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.833745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdaa5-a28e-4a8c-adcc-56be39eeba72 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:48.835130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:48.862140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:48.863101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:48.864148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:48.864148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:48.864148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:48.864148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:48.864830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 279/584] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:48.867232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:48.897435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:48.897757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:48.898329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:48.898329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:48.914431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:48.916117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:48.939620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:48.940699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:48.941677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:48.941677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:48.941677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:48.941677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:48.941677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 306/585] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:48.942336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.073783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c47d21-5049-43f6-9991-6e3677a7aa58 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.076708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:49.148002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.149490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.150011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.150525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.153405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:49.157023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd54af-2f49-4dbe-8c0b-1a896f147590 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.165750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.168603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:49.200971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:49.203041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 280/586] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:49.204835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.227146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.227446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.228084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.228084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.242727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.245256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:49.268786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:49.270886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 307/587] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:49.272841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.404627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872311d3-cb48-429f-ba6b-4dda585334f9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.463507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.464195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.464681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.464958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.465755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:49.466533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.467987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.480307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6b7f-62f1-4840-b841-447fd3900214 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.487137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 281/588] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:49.488939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.535006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.536296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.536881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.537559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.539786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:49.540465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.542956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.543677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:49.569500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 308/589] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:49.569500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.611202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:49.697253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dd89b-e9b2-4287-a4d4-e8deddc1aa09 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.727678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9db8ca-4326-455f-9f4b-a15eef288b89 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:49.759943 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.760659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.760948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.761233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.762181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:49.762504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.764048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.792603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 282/590] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:49.794148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.816096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:49.816891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:49.817275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:49.817688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:49.818832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:49.819094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:49.820832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:49.857047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 309/591] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:49.858536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:49.968865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:50.019230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8fb0-4553-4e73-b07b-f9f02e53c9dc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.033746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:50.077796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633b43f-0844-475f-a328-82b33bfb8823 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.091047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.093007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.093989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.094882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.109420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.113129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:50.141634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:50.142865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:50.144326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:50.147278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 283/592] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:50.148666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.158638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.160125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.160650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.161142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.177054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.179896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:50.209674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:50.210749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:50.212117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 310/593] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:50.213413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.357890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae232ad-4d6a-4d9d-b2d6-470f106b490b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.393385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af84d3-3eb6-4349-a23a-b7c1a240e88c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.430151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.431570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.432081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.432585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.450426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:50.455585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.456082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.457235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.457691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.458152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.458799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.473802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:50.479202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=d73b8801206a4973a97fda3fb00f55c5, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.480969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:50.502409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:50.504205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 284/594] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:50.506586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.510405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 311/595] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:08:50 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:50.512824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.705044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e9e3-50d1-4c96-b257-a171c07d5825 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.712989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d852-d4a1-467c-86bf-e226e32e1d64 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:50.791299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.792476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.792970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.793508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.795687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:50.796282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.796468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:50.797587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:50.798089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:50.798580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:50.799223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.800214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:50.800759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:50.803790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:50.832324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 285/596] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:50.834129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.848791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 312/597] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 25 16:08:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:50.850537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:50.973422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:50.981831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:51.097831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe166-906f-40ea-a30c-8808d8a5de11 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:51.101444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426237-c9e4-48eb-ad12-ab7bcfcd11a9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:51.165825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:51.168522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.169320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.170030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.171634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:51.172554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:51.172719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:51.173571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.173862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.174144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/0c159db46d48429b95e0166596c53d81/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.176315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:51.197098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=0c159db46d48429b95e0166596c53d81, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:51.198955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:51.207862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 286/598] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:51.210191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:51.222692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 313/599] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:08:50 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/0c159db46d48429b95e0166596c53d81/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:51.224345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:51.412208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:51.419139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13495c8-9491-46b4-8b0c-a991846aa654 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:51.438138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576e7b-aa1c-43b8-98aa-2a435d132c78 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:51.469720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:51.518897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:51.520346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.520903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.521585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.526208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:51.527984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.528493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.528991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.537689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:51.544294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:51.554061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:51.557382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2515aaa4-e063-404d-bc2d-a489f87e4e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:51.576542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:51.578332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:51.580073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:51.585541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 314/600] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:51.588165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ef26f-64f0-4fec-a188-d3f3b82c9a89 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.588651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ef26f-64f0-4fec-a188-d3f3b82c9a89 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.589133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ef26f-64f0-4fec-a188-d3f3b82c9a89 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.591925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 315/601] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:51.597505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ef26f-64f0-4fec-a188-d3f3b82c9a89 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b82e6557-f901-4173-b5d9-9e4c69c1ba36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:51.608403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:51.610430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:51.612179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:51.612179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:51.612179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:51.612179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:51.616453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 287/602] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:51.620328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.621252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.622113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.663690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:51.663690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:51.780819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 288/603] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:51.781772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:51.836990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ef26f-64f0-4fec-a188-d3f3b82c9a89 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:51.910632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:51.913043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:51.914060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:51.914825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:51.917106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:51.918695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:51.922860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:51.953594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 316/604] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:51.956554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:51.995770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:51.998303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:52.012087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7ca04-c93f-4fe7-a2f0-2e0896364c29 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.083761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:52.085321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:52.086087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:52.087015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:52.088906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:52.089790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:52.092898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:52.118663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 289/605] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:52.120809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:52.169936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68465-352f-47e9-aefb-bf4aff602730 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.236481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:52.238078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:52.238596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:52.239402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:52.260827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:52.275837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:52.277655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:52.279037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 317/606] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:52.280342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:52.340339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e0b7-3048-4fd3-a0c6-8ca0c0b1361e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.414817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:52.417326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:52.418337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:52.419103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/468fc86f361a4c7c91cacc1d90086176/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:52.452599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=468fc86f361a4c7c91cacc1d90086176, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:52.455046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:52.458353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:52.496218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 290/607] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:08:52 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/468fc86f361a4c7c91cacc1d90086176/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:52.498883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:52.511794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:52.525487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cff4f3-c9d6-44f0-8d25-dfae24fcbda0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.609561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:52.611496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:52.612021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:52.612989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:52.629279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:52.633367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:52.681730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:52.683416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:52.685191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 318/608] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:52.694222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:52.868312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9cbb4-acdd-425c-850f-fa7787b044af None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.936358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:52.937629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:52.937978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:52.938575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:52.939670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:52.940121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:52.942513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:52.950049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69b4bf-e1bb-4444-978e-aad95a30bed2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:52.958850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 291/609] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:52.973386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:53.074116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:53.075082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.075505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.075951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.077187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:53.077753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:53.080450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:53.097919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:53.113389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 319/610] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:53.124808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:53.165027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:53.241021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469572-a950-4cbb-b830-cb066ce9510f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:53.325359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:53.326299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.326574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.326851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.350494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:53.352432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:53.369484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21304e28-dcee-4920-a613-cc0c50bdd3e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:53.389481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:53.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:53.392924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:53.392924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:53.392924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:53.392924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:53.393411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 292/611] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:53.402941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:53.498710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:53.500633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.501127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.501648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/8ff3caa305504fcd888458b91bf1fc5b/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.543488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=8ff3caa305504fcd888458b91bf1fc5b, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:53.545052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf551515-394d-47dc-9a2d-aeacbb334050 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:53.564941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 320/612] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:08:53 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/8ff3caa305504fcd888458b91bf1fc5b/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:53.667017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c8d-6197-43c9-b51a-069854671d30 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:53.711769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:53.762733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:53.763934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.764434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.764936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.766401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:53.766950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:53.769602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:53.781675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7becf9-da0d-47bf-9e63-799a83fbfe4c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.782121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7becf9-da0d-47bf-9e63-799a83fbfe4c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.782301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7becf9-da0d-47bf-9e63-799a83fbfe4c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.784276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 321/613] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:53.790331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:53.790808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:53.791286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:53.791787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 293/614] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:53.801053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:53.817781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:53.819238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:53.893806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:53.928263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 322/615] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:08:53.990287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:54.006464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed5e0f-3fa5-40ae-8a3b-ac719306cc42 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:54.090449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:54.091887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.092382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.092861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.106783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:54.109570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:54.153385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:54.154543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:54.155601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 294/616] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:08:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:54.165361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:54.217238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2c67a-5687-4c50-8cf3-6ffdae36a6db None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:54.241809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:54.315906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:54.316754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.317042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.317325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.346328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:54.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:54.365204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:54.366395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:54.367988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 323/617] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:54.388040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:54.413515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd9196-ed23-489a-8d15-608e56880695 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:54.487687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:54.488911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.489470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.489948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.491128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:54.495648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:54.497760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:54.499336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:54.523949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 295/618] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:08:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:54.532733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:54.688498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:54.705355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f57c-fde0-4ae4-af80-de609553858e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:54.719883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0363d-d90c-4cd4-8d51-dd7be7b30a31 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:54.817517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:54.818968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.819459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.820840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.823294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:54.824480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.824982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.825516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.827060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:54.827621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:54.830387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950d5c-0890-49d3-b1de-6cf12b0f87fb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:54.839474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:54.842489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:54.847505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 324/619] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:08:54.850089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23091f27-00ba-42af-900a-307c00ce987b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.850569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23091f27-00ba-42af-900a-307c00ce987b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.851064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23091f27-00ba-42af-900a-307c00ce987b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.853869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 325/620] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:54.855153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23091f27-00ba-42af-900a-307c00ce987b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-02b29c97-35b2-408b-b1d0-4320a0a30c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:54.882598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:54.883960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:54.884950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:54.884950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:54.884950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:54.884950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:54.890273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 296/621] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:08:54.892916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:54.893435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:54.893918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:54.929136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:54.931551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:55.062351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:55.072066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23091f27-00ba-42af-900a-307c00ce987b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.074868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 297/622] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:55.076632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.142094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.143535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.144014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.144452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.166865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:55.169770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:55.205862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:55.206847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:55.207771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:55.207771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:55.207771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:55.207771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:55.208441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 326/623] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:55.209736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.216429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:55.235568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd450e2-1e49-4f31-ba42-4d71e190db35 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.305521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.306696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.307184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.307686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.309133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:55.309739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:55.312388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:55.335920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 298/624] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:55.337420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.390481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bcb63-24ef-4124-ad24-825af1da6d52 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.441730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.442576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.442857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.443124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.458682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:55.467712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:55.469626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 327/625] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:55.472053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.495007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:55.519340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7c09a-68cc-4088-b211-7158153bc770 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.583427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.584729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.585237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.585777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.587268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:55.587726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:55.590525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:55.603679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:55.607417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2894-aab1-43c5-9b04-aff8a552e2d4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.615157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 299/626] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:55.616519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.669966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.671933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.672762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.673629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.692195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:55.695591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:55.728379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:55.730160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:55.731914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:55.732754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 328/627] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:55.733926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.824409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f003e-ba6d-4df5-8e4c-97e206f0df34 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:55.883596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:55.899855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:55.899855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:55.899855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:55.899855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:55.901253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:55.902749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:55.905233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:55.923202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 300/628] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:55.924853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:55.968385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f4106-8798-4f69-8dd5-a0e4e0242c84 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.036778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.038825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.039336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.039836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.056895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:56.057159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.060247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:56.089649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:56.091844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:56.093439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 329/629] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:56.094453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:56.156903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fa170-c604-4f03-a509-1d67b8a28dec None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.234438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.235579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.236083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.236585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.238207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:56.238768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.241730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82c91-22c7-4bd0-b521-8ebf3e6d1c83 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.267416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 301/630] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:56.269925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03295-7486-4ee1-8290-0f70c8caf5e5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.270425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03295-7486-4ee1-8290-0f70c8caf5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.270918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03295-7486-4ee1-8290-0f70c8caf5e5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.276961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 302/631] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:56.276961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef03295-7486-4ee1-8290-0f70c8caf5e5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:56.315407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdd8d8-d0ef-4157-844d-7d7ab21552f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.374746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:56.383955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.384798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.385080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.385397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.405517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.408471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-dc6d6066-f394-46f5-a684-0bfd8628e59c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:56.443208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:56.444276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:56.445733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:56.445733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:56.445733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:56.445733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:56.445990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 330/632] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:56.448148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537022f-53f5-4c82-8c93-2184965731a9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.448653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537022f-53f5-4c82-8c93-2184965731a9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.449139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537022f-53f5-4c82-8c93-2184965731a9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.451882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 331/633] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:56.453150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537022f-53f5-4c82-8c93-2184965731a9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:56.462843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef03295-7486-4ee1-8290-0f70c8caf5e5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.489846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:56.508294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.508992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.509536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.509665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.510440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:56.510761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.512248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d59430-df81-469e-865c-c8efad2338ca None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.541219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 303/634] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:08:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:56.543872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.544347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.544792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.569960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:56.571644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:56.604804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537022f-53f5-4c82-8c93-2184965731a9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.648179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.649242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.649658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.650030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/4f36fd1f37394b49ac38568200578c32/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.667472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=4f36fd1f37394b49ac38568200578c32, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.668955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b61e0-d0da-4552-84f1-005c9781c3ae None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.674842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 304/635] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:56.676217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:56.691467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 332/636] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:08:56 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/4f36fd1f37394b49ac38568200578c32/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:08:56.693898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.694390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.694793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.725690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:56.727691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:56.780938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:56.853421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 333/637] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:56.853421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:56.886957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534cb1-b09f-409b-a033-5ed7118c26d8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:56.899838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:56.951185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:56.952392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:56.952871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:56.953481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:56.954906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:56.955556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:56.958299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:56.980151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 305/638] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:56.981096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.063506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedc5b1-5c1e-40ac-b322-4f2cbbd9ecaf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.121315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.122544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.122836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.123268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.150412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1196331-f4c6-4830-bfb0-f1337cc54684 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.186972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:57.213842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.215046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.215480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.215911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.234935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:57.243766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:57.245744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 306/639] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:57.247498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.270320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:57.273365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:57.289044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:57.416878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763bfcb-42a0-4bba-bbd5-cd22d908fa42 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.437299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 334/640] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:57.437299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.490923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.492027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.492502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.493012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.510725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:08:57.519698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:57.530631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 307/641] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:57.532487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.574292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:57.648735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da68b39-b019-4c14-8b55-8c58374c449f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.679752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:57.697933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.699136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.699561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.699981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.706155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99e6e-53cf-44fd-b2eb-ad50f272917c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.709967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:57.712282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:57.730613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:57.732861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 16:08:57.735046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:57.737046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 335/642] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:57.740077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.777445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.778394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.778854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.779305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.780482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:57.780983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:57.783153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:57.798710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 308/643] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:57.800002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:57.925977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b492-5a5b-49af-b21b-a7883cd99e87 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.965628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:57.980031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53bc0-5f26-457e-9272-3607a85c1b39 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:57.982482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:57.983210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:57.983501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:57.983768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:57.998194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:57.999099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:58.022083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:58.023063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:58.024273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:58.024273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:58.024273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.024273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:58.025726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 336/644] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:58.026930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.054331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.055436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.055940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.056597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.058311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:58.058855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.061800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.081861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 309/645] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:58.083682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.111933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:58.185787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90115c1-490b-4a23-9df6-41377afda2ac None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.253193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.254099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.254557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.254992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.256441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:58.257309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.259276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f2f3-c292-4d4a-9545-9b1cdfb7fd57 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.260036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.299421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 337/646] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:58.299421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.327930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.328674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.328951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.329219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.343703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.345383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.349756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:58.368318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:58.377925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 310/647] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:58.379708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.469318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884c1c-7dd2-4368-88e8-f6e4f06901e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.478225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:58.537264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.538606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.539046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.539491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.540564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:58.540880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.542827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.555268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875bc63-422a-41b6-954a-9446fa5a85dd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.567542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 338/648] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:58.569373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.633130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.634084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.634590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.635016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.656011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.658479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:58.685418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:58.687330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:58.690069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:58.690069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:58.690069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:58.690069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:58.691741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 311/649] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:58.692757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.759600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:58.775440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551944-8479-488c-b0ec-2b0e11d3f011 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.847994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.849430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.849994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.850408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.894110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758283aa-5da7-4447-b803-fd89de52d915 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:58.928169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:58.964824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:58.966092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:58.966595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:58.967095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:58.968528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:58.969089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.971371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:58.994414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:58.995082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 312/650] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:58.996089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:58.998279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.175205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20484d4-f748-406d-9af3-3ac3a882c34b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.181316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.192469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 339/651] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:08:59.194367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.243300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.244694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.245189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.245733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.259066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.261853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:08:59.284209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:59.286063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:59.286992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 313/652] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:59.287482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.331694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.360630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fdc7-3275-4fef-bcbc-8a5d82a1a6ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.422355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.423310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.423573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7e02-9956-49b1-8c2c-06d611d7218b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.423710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.424274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.425402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:59.425827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.428360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d511375-b544-4bb3-ac07-40e63612e167 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.448557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 340/653] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:59.451236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ddb62-7475-4667-9c16-b66b7ae9a1cc None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.452078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ddb62-7475-4667-9c16-b66b7ae9a1cc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.452767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ddb62-7475-4667-9c16-b66b7ae9a1cc None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.461983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 341/654] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:08:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:08:59.462695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ddb62-7475-4667-9c16-b66b7ae9a1cc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.474387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.475356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.475681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.475970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.485861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.487634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:59.505114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:59.506527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 314/655] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:59.508075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.550758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.628290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.635008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ddb62-7475-4667-9c16-b66b7ae9a1cc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.638288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48e0-5317-4123-90bf-00f0f099f2bb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.679132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.679870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.680287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.680614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.694125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.696218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.696643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.697744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.698464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.698718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.700199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:08:59.700735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.703376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77e9d4-0dda-40f3-9b93-dc28dfb83154 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:08:59.717552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:08:59.718615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 342/656] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:08:59.725366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.726347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 315/657] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:08:59.726347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.726347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.726347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.752510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:08:59.754483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:08:59.838963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 316/658] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:08:59.840378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:08:59.898851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7554-8bb9-4dfb-a134-9645fccb9dc0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:08:59.908169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.946866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:08:59.948182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:08:59.948674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:08:59.949174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:08:59.964432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:08:59.967362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:08:59.993440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:08:59.996155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:08:59.998261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 343/659] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:00.000120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.014803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177c5c-7df5-4469-aeed-dd5b60fb90a4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.065841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.066630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.066974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.067332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.068349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:00.068839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.070588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:00.094901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 317/660] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:00.096412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.167564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea11a4-f52b-4dad-a24f-fe92444ad89e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.230633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.231766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.232272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.232751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.234166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:00.234753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.237395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:00.257517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbec111-b370-444d-a143-af788a339df8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.267151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 344/661] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:00.267151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.306604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:00.319841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.320631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.320941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.321285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.322392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:00.322723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.324408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:00.331733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 318/662] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:00.332809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.352423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:00.447806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c060d0-b24b-4803-b81e-ec2de04c09e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.515684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.516646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.517037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.517556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.523779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db698f9-0e40-4a10-9846-b37b0cedfdab None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.534045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:00.544001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:00.546257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 345/663] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:00.548102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.591023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.591885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.592206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.592561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.595678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:00.595678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.596039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:00.628865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 319/664] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:00.630245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.715604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:00.738720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e33c18-72e1-4961-8cf5-17fe7ef8387f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.761132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:00.805570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.806609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.807113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.807618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.817011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c52b-65b8-40de-a6cf-2764c38b9fc2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:00.863329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:00.864401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:00.864787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:00.865145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:00.877432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.880348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:00.910868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:00.912855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:00.914572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:00.921846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 320/665] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:00.924522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:00.927557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:00.930537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:01.097974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4607-0814-4a02-9a78-b93da18d59f6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:01.100820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.126132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 346/666] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:09:01.131178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:01.147622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.161961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:01.163655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:01.164174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:01.164766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:01.166278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:01.166847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:01.169810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:01.186894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 321/667] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:09:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:01.188619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:01.334260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e3c0-cd02-48fd-87af-11c2e6df68b0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:01.344951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d4016-ad9d-49b8-bce4-80109889c797 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:01.390572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:01.391179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:01.391467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:01.391745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:01.392656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:01.393005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:01.394628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:01.411114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:01.412614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:01.413401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:01.413401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:01.414426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 322/668] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:01.415006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:01.530751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.534239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.557159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:01.559411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:01.634234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7d53-ce2a-4708-ac1e-72fa14f5f52a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:01.690846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:01.691637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:01.691945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:01.692224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:01.715385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:01.718379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:01.748633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 347/669] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:09:01.748633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:01.760376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:01.762526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:01.764343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:01.764343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:01.764343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:01.764343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:01.765586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 323/670] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:01.767106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:01.933618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.936420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:01.941641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19802e-4c73-42bf-b65b-83015dc25f4e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:01.969797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1276af-c63c-4739-bbea-44761020559f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.019275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.021098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.022086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.022803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.024476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=0c159db46d48429b95e0166596c53d81) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:02.025061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.028049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.028428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.030039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.030039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.030628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.053379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 324/671] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:01 2022] GET /identity/v3/role_assignments?user.id=0c159db46d48429b95e0166596c53d81&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:02.053379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.058164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.060836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:02.105198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:02.106726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:02.108934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:02.108934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:02.108934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:02.108934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:02.108934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 348/672] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:02.111329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.281603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64b5c-dd85-4fae-9b2b-dc8c6164ef77 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.327128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484a71-5958-4c0d-89f2-4b3a00f0b927 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.370149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.371454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.372089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.372587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.394734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.395802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.396053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.396326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.396766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.397955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.399121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:02.399644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.402405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.415434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:02.428224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 349/673] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:02.429804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:02.441691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:02.443264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:02.445522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:02.445522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:02.445522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:02.445522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:02.446563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 325/674] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:02.447393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.498800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:02.634169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd848-2fbe-4e56-9247-4e72498d3142 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.654734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db2bf-246c-4189-9898-14e7bae3e142 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.687464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.688357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.688729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.689008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.702906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.704640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:02.725125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:02.726922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 326/675] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:02.728724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.746202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.748141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.749037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.749920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.751837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:02.752808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.756034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.782959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 350/676] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:02.784709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:02.841754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:02.882398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ef33-38b7-4f90-85f7-18e49fc71a4e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:02.917101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:02.953758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:02.954932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.955378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.955820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.957230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:02.957856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:02.960654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e95e-4049-4e4c-becd-dd80f3aea5c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:02.990510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 327/677] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:02.993325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a688-9f08-401f-a44d-af9a5b8d8c8f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:02.994295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a688-9f08-401f-a44d-af9a5b8d8c8f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:02.995068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a688-9f08-401f-a44d-af9a5b8d8c8f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:02.998754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 328/678] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:03.001034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68a688-9f08-401f-a44d-af9a5b8d8c8f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.013182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732aed16-4f9f-4a01-b1e8-c7bdec08a6dd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.081104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.082243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.082744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.083269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.084734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:03.085232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.087991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:03.115556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 351/679] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:03.115556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.198391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68a688-9f08-401f-a44d-af9a5b8d8c8f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.249217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.250698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.251205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.251685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.256273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:03.262711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.265436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3d0007aa-8a5b-4856-a8e2-43e8e66236cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:03.284273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:03.285322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:03.286722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 329/680] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:03.288154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.288154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.288154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.312702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:03.313789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4208bf-f6d4-4afb-9dbe-12b46b499145 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.314049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:03.341779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:03.368868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.370018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.370338 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.370609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.379812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.381551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:03.399781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:03.402502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 352/681] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:03.404255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.443837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 330/682] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:03.445495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.594787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225f666-10c0-4735-8d2f-fd34cc458056 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.651919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.652608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.652889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.653599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.654054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=468fc86f361a4c7c91cacc1d90086176) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:03.654370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.656456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:03.663952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 353/683] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:03 2022] GET /identity/v3/role_assignments?user.id=468fc86f361a4c7c91cacc1d90086176&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:03.664368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fdb5f-1534-443c-a13b-dc3bc0f51a6d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.665449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.711289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:03.725648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.726392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.726699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.726981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.728021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:03.728212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.729772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:03.767550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 331/684] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:09:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:03.768157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:03.792416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:03.888165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40b65-f6e4-401c-ad1b-3477de80a44e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.957277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09372370-154b-40c9-8b56-4d2ac30e127b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:03.958751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:03.960250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:03.960907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:03.962136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:03.965153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:03.965920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:03.968960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:04.000070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 354/685] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:04.001817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:04.044885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:04.046829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:04.047644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:04.048424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:04.075351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:04.093001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:04.095223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 332/686] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:04.096937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:04.205071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:04.259692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e070917-8496-4ad3-baa6-c334246d29a9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:04.267134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:04.331598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:04.332822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:04.333775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:04.334586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:04.347211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef121f9-8492-415e-b39f-84636ce9a632 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:04.442250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:04.442250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:04.442250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:04.442250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:04.444945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=8ff3caa305504fcd888458b91bf1fc5b) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:04.446008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:04.449401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:04.466138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 333/687] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:04 2022] GET /identity/v3/role_assignments?user.id=8ff3caa305504fcd888458b91bf1fc5b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:04.467893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:04.525040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:04.528109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:04.706585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ab9a4-c17b-4c2e-864e-f97741473639 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:04.730431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:04.797305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:04.798539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:04.799449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:04.800679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:04.802674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:04.803266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:04.806485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:04.819171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 334/688] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:04.819784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 355/689] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:09:04.827218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:04.830213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:04.835451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:05.066517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f675-e91b-4fe1-bb3a-92f67a9145d7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:05.125425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6330e-cf4b-4805-942b-55de12b09781 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:05.148849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:05.150479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.150989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.151495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.177187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:05.180482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:05.224315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:05.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:05.227373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:05.229255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:05.229255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:05.229255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:05.229255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:05.229565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.229565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.229565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.229809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 356/690] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:05.237398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:05.280218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:05.283128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:05.363750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:05.366923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:05.369997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:05.369997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:05.369997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:05.369997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:05.369997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 335/691] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 540 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:05.377373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:05.424306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:05.481373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:05.531640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64f74-8b0a-42b1-a117-b8d672c37f8c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:05.614804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:05.615959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.616433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.617192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.619268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:05.619877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:05.622615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cb775-6db4-4f19-92af-5dc47a8b07aa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:05.642156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f5430-1702-44c4-b72d-fb7d75b27fcb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:05.657516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 357/692] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:05.659702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c5e4-7918-4db4-bd52-f6648aa9a7c2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.659995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c5e4-7918-4db4-bd52-f6648aa9a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.660353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c5e4-7918-4db4-bd52-f6648aa9a7c2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.667825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 358/693] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:05.669473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58c5e4-7918-4db4-bd52-f6648aa9a7c2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:05.723233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:05.724555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.725562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.726099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.728442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:05.728989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:05.732594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71678fa6-65e6-4a17-968d-1a76b7b8e2b5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:05.769928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 336/694] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:05.772571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.773225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.773841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:05.808257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:05.810660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:05.902704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58c5e4-7918-4db4-bd52-f6648aa9a7c2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:05.934621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:05.969308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 337/695] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:05.971263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:05.981813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:05.985466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:05.986390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:05.987110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.002186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.006346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.022638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:06.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:06.053863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:06.055465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:06.059771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 359/696] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:09:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:06.066170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:06.194276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26329f-0e1d-4b15-9eac-11cb15734291 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.272314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:06.274121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:06.274614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:06.275200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.286534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a88375-a240-4263-a7a0-1cf05dfcb054 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.290421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.293801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:06.327231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:06.329268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:06.331417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:06.342497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 338/697] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:06.344334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:06.370390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:06.373490 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:06.374367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:06.375063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.405466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.408357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:06.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:06.446229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:06.449126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:06.449126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:06.449126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:06.449126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:06.457944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 360/698] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:06.460434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:06.510977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:06.524645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:06.587638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436fec-01e1-4a3e-b9af-0dfd3fd67513 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.636140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560009b-4669-43d9-8113-b566278efacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:06.637014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560009b-4669-43d9-8113-b566278efacc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:06.637429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560009b-4669-43d9-8113-b566278efacc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:06.637809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560009b-4669-43d9-8113-b566278efacc None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.638775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560009b-4669-43d9-8113-b566278efacc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:06.639183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560009b-4669-43d9-8113-b566278efacc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.640820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560009b-4669-43d9-8113-b566278efacc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.673532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc46a-3657-4224-ae8f-fe9237da4d15 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.685391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 339/699] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:06.685391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560009b-4669-43d9-8113-b566278efacc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:06.754142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:06.755292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:06.755805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:06.756476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.757984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:06.758467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.760429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.784357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 361/700] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:06.784357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:06.876372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560009b-4669-43d9-8113-b566278efacc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.940180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:06.951319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:06.958142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:06.961054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:06.963156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:06.963892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:06.965905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:06.966418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:06.968679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:06.985669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d95df-4b42-40f6-8b37-b48c3838ffd1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:06.997073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 340/701] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:06.998254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:07.070749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:07.072292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.073035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.074045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.193777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e59fed-1b6c-4280-9988-6069aa047635 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:07.226356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:07.228242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffaa-4b8a-40ff-a272-4225e79fd04f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:07.268067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:07.269390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.269883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.270373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.272442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=4f36fd1f37394b49ac38568200578c32) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:07.273220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:07.276494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:07.294936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 341/702] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/role_assignments?user.id=4f36fd1f37394b49ac38568200578c32&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:07.294936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:07.398775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:07.407388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:07.445633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 362/703] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:09:07.447713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1d20-d1f0-46ab-9f48-836802facef1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.448267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1d20-d1f0-46ab-9f48-836802facef1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.448851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1d20-d1f0-46ab-9f48-836802facef1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.455227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 363/704] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:07.456575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1d20-d1f0-46ab-9f48-836802facef1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:07.473646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057c5c-91ce-4f03-87e6-807bf645b5cd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:07.539727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:07.545676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.547844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.548367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.549877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:07.552702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8091b-8e81-44d7-9fd0-ba013ec51d9f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:07.592230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 342/705] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:09:07 2022] POST /identity/v3/users => generated 254 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:07.594824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.595373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.595953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.635008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:07.637138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:07.642166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1d20-d1f0-46ab-9f48-836802facef1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:07.707731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:07.709802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.710093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.710410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.725989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:07.728186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-18ddbf33-af15-4380-b3ae-219c0d8bab91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:07.749393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:07.750377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:07.751354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:07.751354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:07.751354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:07.751354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:07.754056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 364/706] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:07.755898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4bbf-32a6-462b-bc78-fb16eabd28c3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.756489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4bbf-32a6-462b-bc78-fb16eabd28c3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.757327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4bbf-32a6-462b-bc78-fb16eabd28c3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.763604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 343/708] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:07.763604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:07.764096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 365/708] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:07.766201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.766809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.767348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.788460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:07.790454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:07.823836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:07.828281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:07.880135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 366/709] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 16:09:07.881522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:07.932154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22474fc-4f05-41f3-b10a-302edfcd7392 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:07.978116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:07.978718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:07.979104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:07.979441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:07.980220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:07.980567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:07.982168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:07.996505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 344/710] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:08.001813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.033377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeeb127-a059-4788-a550-db879872bf1c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.075628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.076833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.077270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.077754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.078655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.080680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:08.098151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:08.099238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:08.100280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:08.104138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 367/711] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 25 16:09:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:08.105465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.131361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07491b-4608-45ba-bb31-c8e8698ca0be None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.133537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:08.158155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:08.181438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.181848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.182277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.182614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] PATH_INFO: `/v3/services/network` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.183447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.185032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:08.200931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:08.202888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:08.204537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:08.207402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 345/712] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:07 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:08.208554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.319767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb425bb3-62d3-4e58-9e98-5a7dd76571e1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.352284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64609e15-aa2e-4801-af32-739501abbfaa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.370347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.371391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.371808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.372184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.381367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.383439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:08.400845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:08.402688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:08.404588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:08.409485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 368/713] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:08.410590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.434786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.435649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.436026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.436401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.437251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:08.437713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.439342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.449374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 346/714] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:08.449374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.489322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:08.579490 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:08.600772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b60e8-ea61-424c-b068-03c8445bd196 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.633543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320215-fbce-4960-9329-2bfc63247477 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.669891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.670574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.670853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.671128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.671899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] RBAC: Adding query filter params (name=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:08.672207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.674128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.681796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 369/715] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:08.687580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.714691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.715380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.715689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.715929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:08.716738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:08.717047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:08.718653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:08.740382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 347/716] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:08.741704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:08.912307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:08.920171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496f18d-0e54-43c7-9a08-1b790b8e996d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.934764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3d0cd-8ca6-4149-b10a-e87377858d98 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:08.994652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:08.996830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:08.997893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:08.998891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:09.000945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:09.001374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:09.002277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:09.002566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:09.003035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:09.003484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:09.004831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:09.005426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:09.007281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:09.008426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8df5-2300-45dc-9161-f52f91c351dd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:09.020193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 370/718] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:09.022515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:09.025727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 348/718] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:09.257939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:09.427053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273579f-2733-4f52-b2d9-8b18950bb700 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:09.506622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:09.508181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:09.508544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:09.508902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:09.510537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:09.511047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:09.513049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d060e-348e-48bb-96de-b981caafaeeb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:09.529584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 371/719] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:09 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:10.301565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bdaa0-bbc5-4b3c-9a12-b6562a12439a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:10.302388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bdaa0-bbc5-4b3c-9a12-b6562a12439a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:10.302698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bdaa0-bbc5-4b3c-9a12-b6562a12439a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:10.305647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 349/720] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:10.311964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:10.312679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:10.312679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:10.358813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:10.361138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:10.446127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1df-5b0f-47cd-8b3a-88155a9b3cd9 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:10.446632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1df-5b0f-47cd-8b3a-88155a9b3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:10.447222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1df-5b0f-47cd-8b3a-88155a9b3cd9 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:10.455498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 350/721] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:10.459726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:10.460596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:10.461416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:10.528616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:10.532649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:10.568453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 372/722] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:10.604222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:10.637899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:10.712875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 351/723] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:10.767777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:10.881300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b02efa-ac61-442e-9a3e-879dcca0ba9c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:10.931392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:10.932497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:10.934296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:10.934855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:10.964537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:10.983625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:10.984721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:10.985681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:10.989803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 373/724] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:10.995621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:11.029638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:11.083914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d280f6-c5e0-407c-9aac-5b995ca8e094 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:11.174200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:11.189656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:11.190877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:11.192074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:11.193374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:11.223894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:11.232308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75933e-ddcd-4ab3-b807-faaa7d12f3e3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:11.237251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:11.238312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:11.239146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:11.252548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 352/725] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:11.259246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:11.342947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:11.347116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:11.349088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:11.350404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:11.353146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:11.356535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:11.360988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:11.385973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 374/726] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:11.393966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:11.638362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a183-7c9f-4a45-ba79-f935ba9dac44 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:11.727764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c262855-5c62-441d-bf13-b8ae082eea2c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:11.734266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:11.735074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:11.735382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:11.735664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:11.736444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:11.736754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:11.738221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:11.753579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 353/727] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:11.763434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:11.774557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:11.806775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:11.808658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:11.809589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:11.810304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:11.817871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:11.852197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:11.854969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:11.913827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:11.915538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:11.917257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:11.917257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:11.917257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:11.917257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:11.917257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 375/728] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 523 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:11.930459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.015758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417694-bd29-4020-b1b2-7f6e0f8d902c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.091420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.092751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.093237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.093983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.117908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.120631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:12.160518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:12.162399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:12.163906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:12.163906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:12.163906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:12.163906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:12.163906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 354/729] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:12.173141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.217745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1d1f-b960-4c60-a7c9-fb390f6445dc None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.294353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.295178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.295507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.295791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.296578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:12.296888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.298395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82674c2-e708-4075-8edb-ba694956a52f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.320367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 376/730] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:12.321920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcabf7-96b6-44fb-b271-66e7bfe0ddf7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.322198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcabf7-96b6-44fb-b271-66e7bfe0ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.322600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcabf7-96b6-44fb-b271-66e7bfe0ddf7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.324906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 377/731] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:12.325706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:12.332359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcabf7-96b6-44fb-b271-66e7bfe0ddf7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.417909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43cc94-3a4f-4f0d-b59f-09ec66f95d1d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.425956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:12.483294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcabf7-96b6-44fb-b271-66e7bfe0ddf7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.486657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.488211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.488580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.489043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.490552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:12.491155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.493543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b020413-a062-49ef-bf6f-ef7552d5236c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.514735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 355/732] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:12.515563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d31db-d755-4309-9b48-49e26dd68aec None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.515842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d31db-d755-4309-9b48-49e26dd68aec None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.516121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d31db-d755-4309-9b48-49e26dd68aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.538398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0d5d31db-d755-4309-9b48-49e26dd68aec None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:12.539794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0d5d31db-d755-4309-9b48-49e26dd68aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:12.550704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.551953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.552404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.552863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.566386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.568068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-70f6efad-5891-4dad-a09f-9f47ebf4036e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:12.589933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:12.591039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:12.592332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:12.593756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 378/733] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:12.595409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73197-62b1-45ca-8b1e-b08c2150209c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.595703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73197-62b1-45ca-8b1e-b08c2150209c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.595986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73197-62b1-45ca-8b1e-b08c2150209c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.597633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 379/734] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:12.598203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73197-62b1-45ca-8b1e-b08c2150209c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.633138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 356/735] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:12.634586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908db4e-e4c9-45a3-b4d1-bbe9fb9c7c79 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.634954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908db4e-e4c9-45a3-b4d1-bbe9fb9c7c79 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.635247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908db4e-e4c9-45a3-b4d1-bbe9fb9c7c79 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.636633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 357/736] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:12.637947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908db4e-e4c9-45a3-b4d1-bbe9fb9c7c79 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.701060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:12.728865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73197-62b1-45ca-8b1e-b08c2150209c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.743471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:12.767876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908db4e-e4c9-45a3-b4d1-bbe9fb9c7c79 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:12.781443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.782305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.782568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.782845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.794258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.794258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.810237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:12.811128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.811493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.811821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.813145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:12.813539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0ccd2e7c-d247-4f6b-8bb3-2d49f46928bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:12.814308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:12.815614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:12.816559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:12.817106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c50a5-dbb5-4cba-9002-24207975c31e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:12.817238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 380/737] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:12.817871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.818383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.818864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.841143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:12.842406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:12.846733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 358/738] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:09:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:12.847512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:12.847801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:12.848089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:12.876549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:12.878698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:12.925433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 381/739] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:12.926351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:12.981012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 359/740] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 16:09:12.982513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.036152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.075599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.080364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302477f1-f746-41a3-9847-817fdc8757ab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.133668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.134733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.135198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.135645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.136779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.139251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.140775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b184a9-c00b-472f-8d9d-bba026552eee None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.161463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 382/741] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:12 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:13.162985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.226588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.228357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.229488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.229979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.231202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:13.232127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.234660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.262082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 360/742] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:13.263429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.344957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb905ff-2aed-4625-a600-37c1826c1db4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.406794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.407478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.407764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.408046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.409512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.431067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d704a-2f6d-43f4-a445-4349cf174958 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.452980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.488179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.489383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.489383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.489637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.490352 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.491770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.509706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.511246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.525369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 361/743] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:13 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:13.525369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.660046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 383/744] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 16:09:13.661257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.683439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaeb7f-0631-4e7b-91f2-f3dbedf42b80 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.730044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.730960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.731245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.731516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.738113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.749655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:13.758731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:13.759873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:13.760734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 362/745] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:13.762148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:13.767851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:13.808604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce97e36-b9d0-49bd-b9d3-b7c77c697af4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.859421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.861635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.862146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.862616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.898413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553202ee-3263-4185-98b2-85b8fbd0684b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:13.942357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:13.944349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:13.945018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:13.945704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:13.964834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.967982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.970337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:13.971927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:13.994164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:13.995155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:13.996110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:13.996110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:13.996110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:13.996110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:13.998882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 363/746] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:14.000097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.068115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.072270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.151515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd5f-86c8-440a-be95-fb4ce1f7fd55 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.154951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 384/747] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:14.156185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.195281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.196099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.196385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.196661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.197491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:14.197803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.199243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.214388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 364/748] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:14.215411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.327986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b31ae-65cd-4171-b461-71a7dbcca2f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.341653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c644a7-4ac5-486e-8c27-ce83eebaf9eb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.366089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.385810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.386573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.387028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.387621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.389088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:14.389426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.391497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.404510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.405715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.406099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.406561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.414971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 385/749] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:14.417652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.427933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.429522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.444759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:14.457444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:14.458535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:14.459526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:14.459526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:14.459526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:14.459526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:14.460283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 365/750] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:14.461476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.582459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b88ee9-ee23-4244-b501-2c570f9c95aa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.624649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.625955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.626460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.626941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.638857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d1960-f03e-4d89-bc7e-4534bbd2fd66 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.640776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.642653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:14.662353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:14.663332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:14.664221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:14.664221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:14.664221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:14.664221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:14.666673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 386/751] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:14.668000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.681743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.682305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.682589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.682871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.691519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.693103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:14.709282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:14.710297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:14.711186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:14.715676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 366/752] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:14.716249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.749928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.754806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:14.792008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9d05a-87fe-4141-ba26-dd555c95c1d1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.831114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.832171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.832692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.833177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.834233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:14.834746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.836110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d71b-cdd0-4332-9ea4-2885caa16bee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:14.836708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.851149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 387/753] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:14.852426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.878415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:14.879137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:14.879421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:14.879700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:14.880592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:14.880864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:14.884787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:14.900738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 367/754] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:14.901116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:14.980175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cdb2-e6e4-4d60-9b61-d372cd13c557 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.015453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.016773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.025542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.026752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.027283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.027766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.028789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:15.029234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e9842-8e67-4f3e-90b7-1574b2fed297 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.029583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.031467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.046089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 388/755] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:15.047481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.086608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.087540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.087825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.088201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.101168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.103763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:15.125351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:15.126635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 368/756] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:15.128310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.197275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb70ac-8434-4282-bfa5-17f113ab36eb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.237906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.239193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.239719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.240265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.251618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.252789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237459d-2bb7-4814-8110-b39d06f91fce None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.253890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:15.271270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:15.272530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:15.273513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:15.276497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 389/757] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:15.277837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.297977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.298784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.299128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.299341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.300212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=4f36fd1f37394b49ac38568200578c32) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:15.300524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.302241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.317623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 369/758] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/role_assignments?user.id=4f36fd1f37394b49ac38568200578c32&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:15.318611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.342507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.350014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.444620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c26c9-288e-43bf-aebe-75f8670fa3d9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.533705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.534321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.534605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.534878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.535730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:15.536038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.538232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.556173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce84575-44b4-409f-a68c-40ee6605b6f6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.577387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 390/759] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:15.582696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.620696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.622661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.623340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.623967 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.626505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:15.627514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.630315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.660563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 370/760] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:15.674692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:15.808103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366b2d-a3f0-432f-9d07-fbc78873d93e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:15.830867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.879454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:15.881185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:15.882266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:15.883083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:15.884904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=6b4413d1949d4f88935b5f3c39c0ac4b) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:15.885764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:15.888621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:15.893804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:15.900811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 391/761] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/role_assignments?user.id=6b4413d1949d4f88935b5f3c39c0ac4b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:15.988591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ea22-c790-4790-a6b1-427cafd8cd01 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:16.047063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:16.047906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.048281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.048558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.061534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:16.062945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:16.098707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:16.100574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:16.102377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:16.102377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:16.102377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:16.102377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:16.109371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 371/762] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:16.115270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:16.387618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2890a-54eb-4e47-8871-4bc8482b16c5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:16.462304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:16.463021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.463304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.463581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.464527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:16.464835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:16.472983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987475c-f118-4f24-8c44-d34a75e297ee None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:16.496139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 392/763] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:16.504204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:16.532554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3276-06e2-49fc-93ee-a154bdff75a6 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.532965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3276-06e2-49fc-93ee-a154bdff75a6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.533150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3276-06e2-49fc-93ee-a154bdff75a6 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.534841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 393/764] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:16.545255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17533ea7-21c2-4251-997b-d0af2089e1cc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.545255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17533ea7-21c2-4251-997b-d0af2089e1cc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.545255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17533ea7-21c2-4251-997b-d0af2089e1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.579910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-17533ea7-21c2-4251-997b-d0af2089e1cc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:16.581173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-17533ea7-21c2-4251-997b-d0af2089e1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:16.619001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:16.697516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322a007-b719-4fda-8b53-497f33b1919d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:16.697775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 394/765] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:16.700157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8cc6-7ffc-416a-abeb-d899c6696a37 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.700902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8cc6-7ffc-416a-abeb-d899c6696a37 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.701592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8cc6-7ffc-416a-abeb-d899c6696a37 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.704739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 395/766] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:16.712820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.713648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.714353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.728678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:16.750643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:16.752008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:16.752521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:16.753034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:16.760051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:16.762168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:16.765227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:16.767063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:16.784044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:16.786083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 372/767] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:16.789615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:16.909746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 396/768] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:16.909746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:16.954935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd84032-a874-4ee6-9acc-5d8be4abbe0b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.003361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.004766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.005319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.005909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.007185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.008980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:17.024380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:17.025534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:17.026406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.029589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 373/769] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 25 16:09:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:17.030703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.082399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:17.094405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6a4b3-d293-49da-b7f3-fd6e0dd7881d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.148954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.149950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:17.150305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.150794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.151531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.153106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:17.153712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.155489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.176933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 397/770] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:17.178428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.233228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728911b-89ff-413c-81bd-bdf7dfdd6952 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.277056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.278365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.278859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.279366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] PATH_INFO: `/v3/services/network` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.280405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.282272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:17.296637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:17.297804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:17.298658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:17.302012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 374/771] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:17.308540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.379190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcba57-eba0-4ab7-9d91-352fb05d95d9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.442965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46f51-a0bd-49cc-8ca8-08188a23c72c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.454613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.455753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.456240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.456723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.458129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:17.462638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.462638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.477515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 398/772] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:17.479087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.480221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:17.517417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:17.527629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.529681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.530344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.530994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.534876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=d73b8801206a4973a97fda3fb00f55c5, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.538127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.573376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 375/773] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:09:17 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:09:17.573376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.756150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f877f-ab8e-446c-aaf6-8368ac8f890c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.798147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94007a-fa7e-4d88-8e38-19b2e3d2f87b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:17.835582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.836288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.836573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.836849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.837715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] RBAC: Adding query filter params (name=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:17.838028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.839560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35bbf3-14d5-46b8-9be6-92db639e3198 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.845997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 399/774] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:17.847512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9668-f2b0-4b3f-b2f6-720568569f00 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.847975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9668-f2b0-4b3f-b2f6-720568569f00 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.848260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9668-f2b0-4b3f-b2f6-720568569f00 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.854511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 400/775] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:17.856699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.857060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.857320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.876990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:17.878351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:17.878808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:17.878981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:17.879247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:17.880146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:17.880567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:17.881073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:17.884304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:17.894937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 376/776] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:17.896622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.977495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 401/777] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:17.979056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:17.982675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:18.014704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:18.083347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca504c-b87c-4e67-870b-7df36c1ef8f2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.132288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.133254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.133686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.134074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.135276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] RBAC: Adding query filter params (type=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:18.135750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:18.138319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:18.146987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 377/778] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:18.147980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:18.189777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3a03-5a97-4f61-8c73-164d24325c6c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.262286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.264237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.265199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.266069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.268257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] RBAC: Adding query filter params (interface=public, service_id=67d5f7f69e214b3487fb562ff7ea89ae, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:18.269123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:18.272365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:18.293521 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 402/779] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:17 2022] GET /identity/v3/endpoints?service_id=67d5f7f69e214b3487fb562ff7ea89ae&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:18.295605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:18.297954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:18.300775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c301f9-209a-4380-a99a-9a3980904953 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.348937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.350089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.350456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.350804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.367795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:18.378663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:18.380626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:18.382626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:18.385473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 378/780] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:18.388039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:18.469938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:18.500230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2fa04-22e5-4f86-bff8-ce86270cb0eb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.539877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3e8e-a6a9-4fb0-a13c-80d712dd76a9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.546803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.547600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.547944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.548252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.549213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] RBAC: Adding query filter params (interface=public, service_id=9ba733a35f23402785c990eef16fe900, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:18.549567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:18.551426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:18.562603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 403/781] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/endpoints?service_id=9ba733a35f23402785c990eef16fe900&interface=public®ion_id=RegionOne => generated 194 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:18.562603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:18.596690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.601612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.601612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.601612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.601612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:18.601612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:18.630819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 379/782] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:18.632572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:18.850133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3778-aac0-4415-b539-8c1bec247289 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:18.940940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:18.963657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:18.964958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:18.965500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:18.966264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:18.967879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:18.968456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:18.971547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:18.989284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 404/783] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:19.000220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:19.008689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:19.114605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1d84-79cc-4cb3-9029-0bdbf3e38eac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:19.214659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:19.215789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.216253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.216736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.219684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:19.222386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:19.250364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 380/784] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/services => generated 772 bytes in 619 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:19.258237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9dbcd-eb22-47af-b981-5e8df0a745d5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:19.338518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:19.340445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.341225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.342048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.366679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:19.369943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:19.397858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:19.398918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:19.399875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:19.399875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:19.399875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:19.399875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:19.402645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 405/785] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:19.409362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:19.543455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:19.559062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ecf2a-3908-411c-a24d-949941c44984 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:19.603231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:19.603936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.604219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.604493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.605253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:19.605596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:19.607083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a60ea2-85a1-4a1b-9214-f9f80b5cc509 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:19.622129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 381/786] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:19.629879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:19.696456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be29859-eb06-451e-b22d-15a6e6f9e3d6 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.696727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be29859-eb06-451e-b22d-15a6e6f9e3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.697129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be29859-eb06-451e-b22d-15a6e6f9e3d6 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.699398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 382/787] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:19.706868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.707328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.707708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.728351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:19.729707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:19.729851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:19.811042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 383/788] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:19.826911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d54b66-52ec-42f9-abf3-22ce81ebc6e7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:19.839016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:19.903084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:19.904537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:19.905052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:19.905627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:19.925503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:19.929097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:19.939513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:19.964805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94c587-4575-4bde-bc05-46960b88a62a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:19.965485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:19.966670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:19.967706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:19.968339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 406/789] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:19.976425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:20.020839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:20.022785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:20.023312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:20.023794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:20.038062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:20.047190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:20.048215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:20.049103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:20.054349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 384/790] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:20.065547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:20.203894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24ef71-2d5a-4c69-8caa-3d67abab5775 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:20.282815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:20.283981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:20.284486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:20.284970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:20.286398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:20.287423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:20.290087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:20.313856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:20.326916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 385/791] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:09:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:20.338567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:20.398534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96846d32-c858-4d68-9f32-d105bc68b5fd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:20.417766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:20.442718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:20.443970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:20.444380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:20.444758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:20.454452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:20.454452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:20.454452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:20.481386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 407/792] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:09:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:20.484336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:20.636376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fe33f-01d4-42b2-b3c6-1a85eee18548 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:20.713239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:20.714748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:20.715219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:20.715661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:20.737079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:20.740485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:20.776322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:20.778153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:20.779905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:20.779905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:20.779905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:20.779905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:20.779905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 386/793] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 442 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:20.786686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:20.870791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75487d4c-648d-4b5c-940c-3f9b3a576996 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:20.947940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:20.990275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:20.991129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:20.991411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:20.991683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.022310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.023827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.026500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9730-5942-4eab-95b3-4bce24ca2f2c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0f5f64c1-e940-4b9e-9e24-7134b3bb64b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:21.071126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:21.072268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:21.073286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:21.073286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:21.073286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:21.073286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:21.073286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 408/794] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 589 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:21.073886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcadcd4-b5b5-4ee7-83b6-c5222f15d439 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.074049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcadcd4-b5b5-4ee7-83b6-c5222f15d439 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.075523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcadcd4-b5b5-4ee7-83b6-c5222f15d439 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.075929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 409/795] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:21.082415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcadcd4-b5b5-4ee7-83b6-c5222f15d439 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.093606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.094828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.095272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.095806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.097462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:21.097856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.100529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dc408-7113-4296-be0d-81659cc1381c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.125236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 387/796] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:21.127142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.127665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.128143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.130177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:21.155960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:21.157813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:21.235248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcadcd4-b5b5-4ee7-83b6-c5222f15d439 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.253776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 388/797] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:21.255755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.285868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.286614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.287029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.287337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.288180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:21.288497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.290184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.304127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 410/798] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:21.305058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.349931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:21.405500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:21.420313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7319e-d28f-41ae-bf2f-28ecadebf876 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.429646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4acd5-9e55-46ee-8a04-f070834d3d99 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.467011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.468262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.468614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.468955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.471357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.472588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.473070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.473563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.478877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.480535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.488058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:21.496745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:21.497967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:21.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:21.499465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:21.500450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:21.501480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:21.503566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 411/800] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:21.504512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.509377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 389/800] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:21.511673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.630654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821842-492b-4c77-8e1c-67daee1fef1c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.637312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff581bec-4772-4cc2-83c5-f2ee32beca82 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.639189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:21.674189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.675436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.675948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.676479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.682074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.682775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.683267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.683737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.684617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:21.684936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.686410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.687677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.689614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.702801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 390/801] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:21.703733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:21.707504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:21.708441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:21.709557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:21.712036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 412/802] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:21.713143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.717886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:21.840576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7788e75-79df-4dbe-912c-d463ccac83c7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.841611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b47f6a-6ff1-4427-aec0-80d22ab786e8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:21.881717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.882251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:21.883129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.883273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:21.883530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.883884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.884082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:21.884521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:21.885784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:21.886383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.888202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.898675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:21.900211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:21.910852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 391/803] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:21.911814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:21.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:21.933071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:21.934029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:21.934029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:21.934029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:21.934029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:21.935542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 413/804] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:21.940348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:21.951987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.023983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.063190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d090972-0ae0-4ac5-94ee-6af08ff38040 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.092301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618869-bca0-43e1-a8ea-0607c8e51eda None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.120980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.121727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.122013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.122345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.123399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:22.123711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.125719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.146122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 392/805] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:22.147792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.154217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.155136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.155422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.155700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.169355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.170987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:22.192080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:22.193131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:22.194069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:22.194069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:22.194069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:22.194069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:22.194069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 414/806] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:22.194786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.259200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.283834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94d00f-bc82-4640-a3b4-5cf159b03171 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.330817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.331970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.332378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.332742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.334418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:22.335014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.336627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.338522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4698f5-edb7-43f7-bf34-aa38918aa101 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.358199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 393/807] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:22.358715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.359151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.391921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.392939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.393235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.393534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/4f36fd1f37394b49ac38568200578c32/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.395707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=4f36fd1f37394b49ac38568200578c32, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.397389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719e602-2f77-43fa-b981-8c6977a52dcb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.416922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 415/808] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:09:22 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/4f36fd1f37394b49ac38568200578c32/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:09:22.418526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272a4eb-0c9a-4a51-b7ec-81023c58fb18 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.418814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272a4eb-0c9a-4a51-b7ec-81023c58fb18 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.419084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272a4eb-0c9a-4a51-b7ec-81023c58fb18 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.420646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 416/809] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:22.425615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272a4eb-0c9a-4a51-b7ec-81023c58fb18 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.522255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c36c5-7af0-445b-b332-4b57b308e1bb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.572578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.573602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.574037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.574500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.575082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272a4eb-0c9a-4a51-b7ec-81023c58fb18 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.575592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:22.576033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.578251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bd103-7c35-45da-956f-c79552be38ed None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.601387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 394/810] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:22.601387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27566120-5dc4-448c-af8d-7cb7599a36cc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.601387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27566120-5dc4-448c-af8d-7cb7599a36cc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.601387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27566120-5dc4-448c-af8d-7cb7599a36cc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.601387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 395/811] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:22.602722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.603003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.603487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.613319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.636654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:22.638549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:22.647195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.648677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.649265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.649922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.663200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.665646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7b2ffbc2-bb69-41f8-825f-98f9aee02474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:22.692095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:22.693423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:09:22.694519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:22.695306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 417/812] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:22.697424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.697424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.697794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.725894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:22.727788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:22.777371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 396/813] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:22.777641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.777753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.869719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 418/814] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:22.871285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:22.917883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918080d-a0ad-4bcd-bbb3-34c88dabc21e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:22.961083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:22.968423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:22.969255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:22.969653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:22.969897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:22.979083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:22.980411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:22.997305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:22.999068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 397/815] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:23.000773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.063740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0418-aae4-4629-9a27-4fa77bd6726f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.133973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.135675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.136241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.136798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.138533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:23.139076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.142048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.181069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 419/816] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:23.181069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.196215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0ac51-4c46-4d38-81bf-d6870a23764f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.207219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:23.255322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.256699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.257481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.257948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.259121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.261613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:23.283485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:23.285149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:23.286483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 398/817] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 25 16:09:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:23.287939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.372516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda873b-bd6e-47fc-a062-0c80ae3c737c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.383594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:23.423247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.423892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.424196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.424465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.425387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.427413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:23.451590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:23.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:23.455520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 420/818] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:23 2022] GET /identity/v3/services/network => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:23.457140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.476678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89285-424f-446c-8f31-4ab3f86883fd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.548288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.549554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.550054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.550676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.551800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:23.552254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.554183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.577392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 399/819] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:23.577392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.613551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:23.643135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c5136-2f67-414e-89a8-8e8b548a75a1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.691151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.692636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.693194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.693723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.736114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193f52-6220-4a0c-9640-15980e16f89e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:23.761198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:23.784590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:23.785704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:23.786113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:23.786435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:23.789798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:23.790210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.792713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.797209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:23.799020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:23.800472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 400/820] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:23.801969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.904565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:23.929809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 421/821] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:23.930892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:23.965745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e229d-166f-498f-b8b3-ac118427fc23 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.010329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.011024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.011309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.011589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.012353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] RBAC: Adding query filter params (name=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:24.012665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.014129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.021404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 401/822] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:24.021404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.076332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:24.089678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2e1f5-eab6-4cbb-ac86-75d3f2061a70 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.138523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.140437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.141127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.141660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/6b4413d1949d4f88935b5f3c39c0ac4b/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.152919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5c056-d331-4b51-b9ae-976d4b633bbd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.170199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=6b4413d1949d4f88935b5f3c39c0ac4b, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.172065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.201371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 422/823] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:09:23 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/6b4413d1949d4f88935b5f3c39c0ac4b/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:09:24.201371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.218109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.219605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.220193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.220766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.222539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:24.223297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.226206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.239350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 402/824] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 25 16:09:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:24.240438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:24.243149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.392491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c6cfd-09c4-4582-ba12-d07cc421e841 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.394291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c6812-850f-4967-abcc-bf630037c905 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.407954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:24.453389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.453389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.453389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.453389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.454058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] RBAC: Adding query filter params (type=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:24.454058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.454058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.460552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 403/825] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:24.462323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.463688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.464189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.464693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.466203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.493075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.496014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:24.533623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:24.535322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:24.537109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:24.537109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:24.537109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:24.537109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:24.537109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 423/826] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:24.538743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.609725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0fd8-378a-469b-9f87-0ecad383e1df None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.671125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.671815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.672099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.672362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.673236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.674708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.690220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:24.725377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 404/827] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:24.725377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:24.744533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ba0ac-d493-4ca8-bee2-dda03cee0abf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:24.810401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:24.815164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:24.817543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:24.818161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:24.820873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:24.822123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:24.827108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:24.886119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 424/828] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:25.034161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b844571-9ee1-4c44-aa9c-da412a6a70b9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:25.117457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:25.117457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.117457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.117457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.117863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:25.117863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:25.119087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c01f9c-a244-4bb9-b349-d396035e6520 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:25.137285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 405/829] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:25.144461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:25.377830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bbce-ca0e-44cb-9861-c476c889462a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:25.391931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a4c3-35fc-4add-80d4-ce35f3d5422b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.392203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a4c3-35fc-4add-80d4-ce35f3d5422b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.392483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a4c3-35fc-4add-80d4-ce35f3d5422b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.398379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 406/830] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:25.404759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.405168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.405508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.442297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:25.443300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.443600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.443902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.446426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:25.453010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:25.453273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:25.454962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:25.457722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:25.473463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:25.475320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:25.477105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 425/831] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:25.484436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:25.582398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 407/832] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:25.613142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:25.634998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:25.647040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa82e4a-a13b-4665-84bb-2e684faed205 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:25.695188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:25.695876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.696159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.696464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.697601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:25.697914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:25.699874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:25.720855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 426/833] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:25.727280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:25.796791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d74a2-7055-46ab-8c1b-08c7485b249d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:25.808467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:25.846132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:25.846950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.847227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.847590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.874285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:25.883017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:25.885720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:25.887627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 408/834] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:25.893739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:25.927634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67301b5-82eb-44a5-8f11-42492265eb4e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:25.995558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:25.996990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:25.997568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:25.998080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:25.999814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:26.000394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:26.003376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:26.021580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 427/835] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:25 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:26.025791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:26.036217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5617-965f-4135-888b-400ef007a13c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:26.078256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:26.079009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.079291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.079674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.080488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:26.080795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:26.082383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:26.093386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 409/836] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:26.099436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:26.280399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b989b-2aa4-4235-8b53-87b8ff2eae77 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:26.301398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:26.325445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:26.326388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.326753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.327035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.344252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:26.346780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:26.379456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:26.381283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:26.382948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:26.382948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:26.382948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:26.382948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:26.382948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 428/837] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:26.392035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:26.595648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e92c6-6567-46f7-ba3f-4120b08de669 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:26.635336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:26.636114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.636399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.636671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.637447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:26.637755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:26.639191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e0c47-46d9-4385-9cb8-e663b9c0104d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:26.642256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:26.657787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 410/838] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:26.660787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:26.685651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7efda-a0ec-41f6-ad8f-ff336bc2d226 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.686159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7efda-a0ec-41f6-ad8f-ff336bc2d226 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.686656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7efda-a0ec-41f6-ad8f-ff336bc2d226 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.689707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 411/839] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:26.697761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857232a-71a9-47a0-8858-efc084269862 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.698217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857232a-71a9-47a0-8858-efc084269862 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.698693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857232a-71a9-47a0-8858-efc084269862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.724951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9857232a-71a9-47a0-8858-efc084269862 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:26.726613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9857232a-71a9-47a0-8858-efc084269862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:26.812488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 412/840] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:26.821380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed277-8000-4ee4-8b7e-829b14df1a72 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:26.854214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857232a-71a9-47a0-8858-efc084269862 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:26.869915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:26.871805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:26.872335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:26.872823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:26.885252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:26.887409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:26.907742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:26.908824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 16:09:26.909795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:26.911985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 429/841] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:09:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:26.918897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:26.982609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.010253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857232a-71a9-47a0-8858-efc084269862 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.042248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.053554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.054957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.055467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.055943 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.071235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:27.080938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:27.081992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:09:27.082861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:27.086397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 413/842] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:09:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:27.094476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:27.134223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952765f-1544-4c66-8566-5b1aca199df0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.202361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.203524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.204022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.204574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.206349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:27.206898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:27.209895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:27.245349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117f25b-7ed8-4230-b2f7-f8201940a28b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.250945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 430/843] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 25 16:09:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:27.254553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:27.299538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.300854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.301323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.302249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.303328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:27.304404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:27.306437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:27.318751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 414/844] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:09:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:27.331592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:27.399881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.404638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.425683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b69915-dc9f-4dc7-84df-c1d1a071371b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.494133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.494964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.495254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.495537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.498095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881885-2de9-42c8-bd1d-c96bef427346 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.542780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.544357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.544602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.545086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.568668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:27.571793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:27.603362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:27.604776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:27.606109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:27.606985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:27.606985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:27.606985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:27.606985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:27.607669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 415/845] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:27.608469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88f12a-15a6-4578-8bca-4f8e3ce19472 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:27.616889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:27.723116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.742150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 431/846] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:27.744927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c513-940a-4251-8f65-224c787063ba None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.745446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c513-940a-4251-8f65-224c787063ba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.745934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c513-940a-4251-8f65-224c787063ba None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.748754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 432/847] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:27.755961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33c513-940a-4251-8f65-224c787063ba None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:27.802504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:27.805843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49528c7-e108-45ac-8db7-f0fecb19cabe None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:27.849682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:27.850390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.850667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.851034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.851781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:27.852092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:27.853530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8682bfd-7876-49bf-96f9-89b9805f76fc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:27.877387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 416/848] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:27.877387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:27.877387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:27.877387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:27.900287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:27.902321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:27.975708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33c513-940a-4251-8f65-224c787063ba None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.041403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 417/849] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:28.041403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.056467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.059436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.059923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.060392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.083353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.085361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:28.107705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:28.109499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 433/850] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:28.111299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.185522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:28.199118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:28.213076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5ae9-a3db-42e2-8b68-af834aa5179a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.269959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.271085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.271502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.271918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.282218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.284227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.288771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a72c-225c-44ef-bf89-1121619fd8f9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:28.302668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:28.304487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 418/851] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:28.306850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.356773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.358068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.358498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.359076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.360124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.361774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:28.377224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:28.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:28.387811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 434/852] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:28.389518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.507457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7baa0-8550-40c0-a713-fa882e28dff8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.556566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcfb67-47b9-4c35-9e3a-9508dabcd2a5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.558764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.559932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.560437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.560919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.562500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:28.563057 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.565663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.569179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:28.598160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 419/853] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:28.599751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.612053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.613197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.614442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.615020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.616651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:28.617129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.619771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.623130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:28.649321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 435/854] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:28.650902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.791081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde54a1-2226-444a-ad89-b3085b00bcda None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.807401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06847-1099-4d91-90cf-7b10526c08f2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:28.834211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.834905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.835217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.835488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.836210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:28.836518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.838259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.848300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 420/855] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:28.849793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.863926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:28.864795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.865076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.865386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.873561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:28.875117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-19c62957-cce8-47ff-aaec-78e869be1fac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:28.890721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:28.891957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:28.893047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 436/856] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:28.894056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae03a-d25a-475b-a50a-08e37bff6406 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:28.894148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae03a-d25a-475b-a50a-08e37bff6406 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:28.894429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae03a-d25a-475b-a50a-08e37bff6406 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:28.895894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 437/857] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:28.896687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ae03a-d25a-475b-a50a-08e37bff6406 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:28.942244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:28.987256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:29.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079c3d-b5db-4144-9eae-27bb15952aee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.061501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ae03a-d25a-475b-a50a-08e37bff6406 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.067805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.068950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.069478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.069950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.102341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.103092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.103374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.103652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.104405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:29.104778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.106246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.112070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 438/858] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:29.113152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:29.172076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.175868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227d35b-0439-486f-bf1a-bec6cf935e57 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.299560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:29.325042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:29.330284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886a2b7-bc09-473b-b062-36c18a077541 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.375405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 421/859] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:29.377699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.378088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.378483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.397253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.398507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.398991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.399529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.401048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:29.401664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.404450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.405484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:29.407497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:29.433320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 439/860] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:29.433320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:29.539809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 422/861] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:29.543682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:29.614771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c701c-6a6a-4124-87c2-4db1f5bd13e1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.666975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.667830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.668112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.668383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.683075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.684832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.693266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:29.711685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:29.712656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:29.714507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:29.714507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:29.714507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:29.714507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:29.714507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 440/862] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:29.715645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:29.724580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864826a9-de9c-4843-a8ba-e8efb2d25e97 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.727341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:29.771214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.771753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.772031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.772316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.773476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=4f36fd1f37394b49ac38568200578c32) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:29.773636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.775149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.791489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 423/863] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/role_assignments?user.id=4f36fd1f37394b49ac38568200578c32&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:29.791489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:29.910164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2015fa-339b-48e5-9432-24749627d847 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.964630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:29.965912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:29.966270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:29.966643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:29.967736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:29.970689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:29.982340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea847a-51f0-4a92-8d57-d3c4dc8e0c30 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:29.992132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:29.993144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:29.994022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 441/864] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:30.000558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:30.042270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.043334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.043867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.044395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.045140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:30.045487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.046927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.066052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 424/865] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:30.070365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:30.096154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:30.125115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:30.193066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cf85e-f607-42fb-985b-55f57c7fb63f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:30.225799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da662d-a78a-47c5-a3cf-5328513ad2e1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:30.244207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.245037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.245549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.245834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.246571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:30.246883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.248418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.263048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 442/866] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:30.264782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:30.270505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.271381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.272683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.272984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.281757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.284197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:30.303661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:30.304601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:30.306010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 425/867] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:30.314450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:30.455176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:30.470576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14375c-ceda-4815-a41a-c8f6447668dd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:30.472425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf393-7276-4389-9931-6285349cac1a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:30.502351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:30.514103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.514763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.515442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.515575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.515738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.515997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.516409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.516907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.517017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:30.517287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.518254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:30.518757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.518855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9670b-9e75-4032-9ab0-470a28171ffe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.520450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.529761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 443/868] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:30.533744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 426/869] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:30.539621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:30.680784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf20c-c4ea-4614-bb8c-da137485ae87 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:30.721868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:30.722518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:30.722852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:30.723188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:30.724082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=6b4413d1949d4f88935b5f3c39c0ac4b) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:30.724391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:30.726042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe0297-f152-4701-a6bc-ea2d343e292c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:30.733530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 444/870] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:30 2022] GET /identity/v3/role_assignments?user.id=6b4413d1949d4f88935b5f3c39c0ac4b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:31.622114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedca6-b913-408e-a711-9d0ebe8447ce None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:31.622949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedca6-b913-408e-a711-9d0ebe8447ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:31.623104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedca6-b913-408e-a711-9d0ebe8447ce None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:31.626090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 427/871] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:31.639368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377460-ad82-4602-b920-d1ad086b9357 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:31.639767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377460-ad82-4602-b920-d1ad086b9357 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:31.641010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377460-ad82-4602-b920-d1ad086b9357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:31.665826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d9377460-ad82-4602-b920-d1ad086b9357 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:31.667260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d9377460-ad82-4602-b920-d1ad086b9357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:31.752788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:31.776652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 445/872] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:31.817530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedca6-b913-408e-a711-9d0ebe8447ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:31.960458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedca6-b913-408e-a711-9d0ebe8447ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:32.001030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:32.001783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:32.002068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:32.002346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:32.003178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:32.004559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf4d67-348c-466a-8f29-b641cad4a902 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:32.026019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 428/873] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:31 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:32.437656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c66e-4d22-4f6d-a784-79827ecc4f6c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:32.438484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c66e-4d22-4f6d-a784-79827ecc4f6c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:32.439207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c66e-4d22-4f6d-a784-79827ecc4f6c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:32.445489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 446/874] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:32.454099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:32.455050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:32.455873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:32.478725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:32.480574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:32.483735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:32.632558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 429/875] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:32.672063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c66e-4d22-4f6d-a784-79827ecc4f6c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:32.891362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c66e-4d22-4f6d-a784-79827ecc4f6c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:32.970665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:32.972451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:32.973412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:32.974374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:32.975914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:32.978900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-035a637d-ff2f-4701-8619-a7534fe04a18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:33.003470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:33.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:33.008059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:33.013378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 447/876] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:33.023542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:33.171189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848a11-b3e3-4bd6-8750-221f81414d6f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.172161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848a11-b3e3-4bd6-8750-221f81414d6f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.172949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848a11-b3e3-4bd6-8750-221f81414d6f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.181531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 448/877] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:33.186758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.187690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.188459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.222348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:33.225292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:33.228258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c933-481a-4c0b-a7a3-9b0cd110f249 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:33.310729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:33.311885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.312358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.312841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.314218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:33.314758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:33.318764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:33.320986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:33.325108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:33.336047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 430/878] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:33.341937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:33.372752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 449/879] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:33.407076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:33.497885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c2407-017c-4af6-8a7c-c3e892c09a0d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:33.543484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:33.544169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.544457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.544732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.545509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:33.545827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:33.547277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb709-0947-404e-a680-8e52ce863b35 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:33.560976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 431/880] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:33.560976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e9f-5301-4d15-acac-3b282eb045cf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.560976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e9f-5301-4d15-acac-3b282eb045cf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.560976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e9f-5301-4d15-acac-3b282eb045cf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.560976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 432/881] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:33.566797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.567066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.567365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.591012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:33.592328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:33.634166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c26750-161b-4e35-a46d-79e5d371dc48 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:33.710259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:33.711441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.711906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.712841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:33.713630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.715474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:33.718278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:33.746004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 433/882] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:33.757975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 450/883] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:33 2022] POST /identity/v3/services => generated 269 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:33.768208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:33.914279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d73e83-7450-491d-8710-a2e169ff8313 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:33.967218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:33.968031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:33.968375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:33.968762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:33.984337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:33.993612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:33.995446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:33.997103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 434/884] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:09:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:34.009770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:34.067783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:34.159885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b84d4-8f15-4745-9ba5-7e8edf8fbaae None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:34.207812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:34.208621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.208901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.209177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.210007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:34.210316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:34.211774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:34.222384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 451/885] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:09:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:34.231495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:34.303521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:34.376200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41f1b-e59e-40b5-a80a-3c9384f6bf98 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:34.417707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:34.418594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.418885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.419163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.432654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:34.434254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c1650a8a-d911-42c2-93fe-838617b0cc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:34.455404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:34.457156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:34.458928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:34.458928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:34.458928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:34.458928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:34.458928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 435/886] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:34.469684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:34.672014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:34.678532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906522a-9d19-48b7-b414-e181a7ce2f28 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:34.683112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf7fe-7563-436b-b728-a7864319e500 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.683599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf7fe-7563-436b-b728-a7864319e500 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.683834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf7fe-7563-436b-b728-a7864319e500 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.685910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 436/887] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:34.692550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.692902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.693251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.718150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:34.719409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:34.746295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:34.747452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.747936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.748400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.749767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:34.750299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:34.752864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:34.784102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 452/888] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:34.785935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:34.810017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 437/889] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:34.832078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:34.866802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:34.938648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b52e-a6df-401d-b3e3-7b9638d2b9b0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:34.949762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dbf5b-a34c-4bf2-924d-66a076d2107e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:34.982998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:34.984318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.984879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.985426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.991841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:34.992828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:34.993121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:34.993389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:34.994084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:34.996034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:34.997005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:34.999208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:35.010282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:35.011354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:35.012226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:35.017008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 438/890] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:35.018027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:35.025810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:35.027155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:35.028130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:35.031279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 453/891] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:35.037412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.078091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:35.170748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:35.182459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb615-40bb-4efc-8e84-ac4e48e6266e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.183053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a54d9-3ee1-4f02-a893-43f7ff9e0aeb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.225952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.227329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.227852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.228355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.229469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:35.230211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.233661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.247367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.248391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.248900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.249445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.250812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:35.251378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.254074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.264619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 454/893] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:35.266858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.269468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 439/893] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:35.274433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.369969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:35.392696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0d9b6-a58f-44e4-a4ec-d2043e412640 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.444323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.445475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.445964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.446529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.474054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53621a5b-3a67-4791-8a09-d1c654161c9c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.543869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.545187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.545874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.546527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.547128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.548222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:35.548913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.549391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.552230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.569905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 440/894] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:35.573308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.625150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:35.665686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:35.696536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 455/895] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:35.706593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.788624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060dee-3214-49de-a58b-3e1a0da38bcb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.839032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df5944-0661-4991-bca0-86e6dd0c9cf6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:35.859139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.860971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.861816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.862554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.864561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] RBAC: Adding query filter params (interface=public, service_id=3b388f90a87d4ca7881f526c396cc7cf, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:35.865445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.868506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.880755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:35.881632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:35.882259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:35.882536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:35.889861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 441/896] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/endpoints?service_id=3b388f90a87d4ca7881f526c396cc7cf&interface=public®ion_id=RegionOne => generated 194 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:35.893687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.896737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:35.898501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:35.918523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:35.920196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:35.922463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:35.922463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:35.922463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:35.922463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:35.922463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 456/897] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:35.926312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:35.966633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:36.054203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee63b-0135-403e-974f-ffe6d1e0ceeb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.083514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:36.094632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.095423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.095688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.095960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.096710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:36.097018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.098617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.117854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27402fe-384b-4c29-a0fa-6cf6710685b9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.120125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 457/898] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:36.123121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:36.190709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.192524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.193486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.194276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.196571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.199888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e5433-b160-49d4-8917-f20b6fb743e5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.217471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 442/899] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:35 2022] GET /identity/v3/services => generated 1268 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:09:36.235094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:36.249006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48936-138b-4390-83e2-96c2b28edcbf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.292392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.293293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.293619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.293895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.302614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.304168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.311706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907538-afa7-4c3d-8ecc-d6c7a68de55b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.312135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907538-afa7-4c3d-8ecc-d6c7a68de55b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.312373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907538-afa7-4c3d-8ecc-d6c7a68de55b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.314267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 443/900] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:36.320508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.320818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.321124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:36.321265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:36.322766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:36.324487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 458/901] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:36.329079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:36.345713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:36.346968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:36.464266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059acc9-1012-4f30-ae69-a237c2410183 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.480193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 444/902] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:36.512045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:36.528062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.529190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.529745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.530202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.531564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:36.532301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.534757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.557776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 459/903] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:36.564373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:36.583440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:36.648361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:36.686390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508a255-ecd0-4269-b151-5037b45d2331 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.733702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba336-ad97-49f8-97a3-6fe652f239ef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:36.749207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.750310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.750755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.751208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.752395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.754824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bea2d-bb2b-4bb7-a19b-68da9819e572 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.783168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 445/904] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:36 2022] POST /identity/v3/services => generated 241 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:09:36.786042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b330-c7ee-4f89-b603-e20a310e9d27 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.786569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b330-c7ee-4f89-b603-e20a310e9d27 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.787020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b330-c7ee-4f89-b603-e20a310e9d27 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.795963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 446/905] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:36.798631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.799078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.799503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.833767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:36.835617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:36.835966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:36.836865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:36.837850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:36.838019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:36.840093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] RBAC: Adding query filter params (role.id=d2864cce36cf4108a69a08feb8529ea6, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=6b4413d1949d4f88935b5f3c39c0ac4b) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:36.841180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:36.844499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:36.858350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 460/906] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:36 2022] GET /identity/v3/role_assignments?user.id=6b4413d1949d4f88935b5f3c39c0ac4b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=d2864cce36cf4108a69a08feb8529ea6 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:36.984439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 447/907] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:37.024569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:37.171548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:37.231301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21c7f-f12a-4bfe-b176-efcc16106ede None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:37.297304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:37.298505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:37.298792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:37.299072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:37.299856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:37.302278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:37.326273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:37.328387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:37.330252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 461/908] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 25 16:09:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:37.336323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:37.438970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:37.487381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf9c10-a167-49ef-ae8f-571a0ae32f7c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:37.532968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:37.533711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:37.534094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:37.534329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:37.535033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:37.535346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:37.536876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:37.543464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 448/909] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:37.549529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:37.679260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9514a0-1817-4f5a-ac51-fb120d8f20e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:37.722159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:37.722881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:37.723159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:37.723438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:37.724281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:37.724593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:37.726261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:37.728149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:37.734140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 462/910] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:37.740529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:37.866322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ee62a-2f97-47ca-a1b1-8672417a1173 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:37.919275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:37.919966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:37.920239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:37.920523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:37.921369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] RBAC: Adding query filter params (interface=public, service_id=0672d2682fd34df6812b97b0ddcbc11a, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:37.921797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:37.923512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c68592-3888-4040-bf1a-880b8fecb4ef None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:37.925785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:37.936667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 449/911] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:37 2022] GET /identity/v3/endpoints?service_id=0672d2682fd34df6812b97b0ddcbc11a&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:37.938177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:38.068588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a2-17ff-4f8b-82c3-53d2ce592a24 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:38.109606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:38.110312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:38.110593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:38.110871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:38.111664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:38.113326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8697c-2a6e-49f4-b884-f41f97bdcf96 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:38.123373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 463/912] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:37 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:09:39.226145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36a91-03ab-4a74-afe3-c326d3e77817 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.226745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36a91-03ab-4a74-afe3-c326d3e77817 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.226745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36a91-03ab-4a74-afe3-c326d3e77817 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.228389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 450/913] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:39.234780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.235879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.236522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.258470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:39.260389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:39.335051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:39.347099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 464/914] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:39.375174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f36a91-03ab-4a74-afe3-c326d3e77817 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:39.506580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f36a91-03ab-4a74-afe3-c326d3e77817 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:39.551456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:39.552931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.553460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.553945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.554890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:39.556849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d2225058-dc9d-4daf-b997-009d54e02696 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:39.571203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:39.572413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:39.573268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:39.576611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 451/915] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:39.584580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:39.707120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f932c58-13d9-4277-8616-32d53fec1cd9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:39.710720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eef4d-b91c-4ac0-96ec-68f8f3854b66 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.711217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eef4d-b91c-4ac0-96ec-68f8f3854b66 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.711692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0eef4d-b91c-4ac0-96ec-68f8f3854b66 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.714684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 452/916] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:39.724082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.724611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.725273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.750156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:39.750947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.751233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.751510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.752280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:39.752578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:39.754129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:39.763536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:39.764620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 465/917] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:39.766279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:39.771381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:39.844669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:39.856588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:39.859399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 453/918] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:39.899292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:39.900729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fada1f1-2b7a-48b7-9555-e95b9b2513aa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:39.944861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:39.945597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:39.945881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:39.946160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:39.947062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:39.947423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:39.948852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:39.955785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 466/919] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:39.962289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.125016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35dc4-01fe-4e3c-82a9-878fed3c0073 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:40.140145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c1119-6487-4333-a784-b142cb87c7ea None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:40.171210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:40.202253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:40.204029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.204912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.205650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.210879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] RBAC: Adding query filter params (interface=public, service_id=d4c65c7f43774a8792637c7e1a4723f7, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:40.211446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:40.214056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:40.218597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:40.220530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.220726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.221179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.222503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:40.225177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:40.227989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 467/920] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:39 2022] GET /identity/v3/endpoints?service_id=d4c65c7f43774a8792637c7e1a4723f7&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:40.237782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:40.251031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:40.253454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 16:09:40.255109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:40.261988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 454/921] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:40.264276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.347980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:40.425975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0743a-fa89-4da5-ba9c-439068c44c32 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:40.437090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4381b-45ef-464b-bf05-1d097c554cd2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:40.484473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:40.485017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.485368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.485659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.486397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:40.486786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:40.488561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056ca91-ace6-4284-bdf7-b6a6b063667a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:40.504854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 455/922] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:40.504854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289114b-0653-4974-9cec-7de6b7d477b4 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.504854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289114b-0653-4974-9cec-7de6b7d477b4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.504854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289114b-0653-4974-9cec-7de6b7d477b4 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.505533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:40.505695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 456/923] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:40.505830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.505830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.506100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.507539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289114b-0653-4974-9cec-7de6b7d477b4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.507718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:40.510477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5247a-9e38-428c-81bc-e03c0e480bd0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:40.524706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 468/924] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:40 2022] GET /identity/v3/services => generated 1497 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:09:40.527195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cbed-f01d-478b-ace7-f434c3471ceb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.527671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cbed-f01d-478b-ace7-f434c3471ceb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.528154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cbed-f01d-478b-ace7-f434c3471ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.562826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cb82cbed-f01d-478b-ace7-f434c3471ceb None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:40.564392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cb82cbed-f01d-478b-ace7-f434c3471ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:40.643465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:40.689309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 469/925] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:40.691033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790fbe-dfd4-4cf3-8726-c626b93395f1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.691464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790fbe-dfd4-4cf3-8726-c626b93395f1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.692080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790fbe-dfd4-4cf3-8726-c626b93395f1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.701545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 470/926] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:40.704237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.704699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.705211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.732247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289114b-0653-4974-9cec-7de6b7d477b4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:40.739626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:40.741631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:40.770851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:40.797169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:40.798285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:40.798752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:40.799247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:40.800291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:40.800693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:40.802972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:40.813550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 457/927] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 25 16:09:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:40.813550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.861567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 471/928] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:40.862499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:40.986605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71f2fa-8fc3-4fa6-a5c6-9e8d494dc4b0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.042918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deaf341-bd34-479f-8181-18fd982a3250 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.055557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:41.063580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.065264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.065795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.066271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.088177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:41.102311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.103480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.103986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.104495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:41.105094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:41.106084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:41.106932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:41.107586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:41.108702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:41.118289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 458/929] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:09:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:41.118289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.157383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 472/930] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:41.157383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.257933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:41.373870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0007-42b7-4776-8420-1c2ac122cc34 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.397377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aedc5-520e-4585-a3fe-49becf39d7ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.453517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.454447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.454906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.455316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.456077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:41.458175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:41.467957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.469389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.470030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.470596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.472141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:41.472774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:41.475711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:41.476045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:41.478718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:41.481486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 459/931] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:41.484672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.497702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 473/932] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:41.506514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.543058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:41.653495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522888b-9fe5-49f8-b835-7c875f42e8fb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.701621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.701621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.701621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.701621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.705521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1964c-202d-4943-8ef6-c221706a52e1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:41.715887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:41.717636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:41.725153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:41.742486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:41.744177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:41.746070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:41.746070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:41.746070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:41.746070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:41.749473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 474/933] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:41.755268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.787192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:41.788289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:41.788736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:41.789180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:41.790607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:41.791109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:41.793874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:41.803458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 460/934] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:41.811415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:41.912714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:41.966036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb33382-8944-42ed-98f5-b63ec0fb373c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:42.027824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95fe5-4c83-458a-ac06-57fee9e704e3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:42.033466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:42.038768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:42.039312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:42.039849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:42.042057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:42.042671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:42.045302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:42.073627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 475/935] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:42.079374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:42.102290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:42.104015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:42.104852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:42.105748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:42.107312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:42.108149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:42.111179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:42.121636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 461/936] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:42.128658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:42.194928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:42.287353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f60ab-16b8-4d76-a8f6-ec7dfab57723 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:42.315024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f32be-139e-464b-bbe6-c8bfbd2d5255 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:42.354957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:42.356730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:42.357632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:42.358487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:42.372865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:42.375980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:42.376620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:42.377646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:42.377821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:42.378120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:42.378565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:42.380111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:42.383096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:42.412570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:42.414985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:42.416790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:42.421958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 476/937] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:42.425648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 462/938] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:09:42.432880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:42.646607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebce51-e039-4629-979a-463c24c6519c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:42.724699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:42.726494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:42.727341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:42.728072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:42.730217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:42.731100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:42.733387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:42.762845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 477/939] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:42.772382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:42.964178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:42.985839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ba01-71e0-402d-a6fc-40b13378de68 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:43.054005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:43.055036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:43.055529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:43.056003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:43.078479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:43.081478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:43.117509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:43.119228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:43.121375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:43.121375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:43.121375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:43.121375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:43.121375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 463/940] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:43.131603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:43.234641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:43.336953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533ba0-e5d6-4476-b503-b223cec495c3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:43.405907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:43.407067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:43.407561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:43.409492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:43.411041 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:43.411634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:43.414378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f31fb-aa53-4b8e-8658-dfdfd2422208 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:43.445420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 478/941] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:43.451243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:43.525497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53b244-f46b-4f50-bfa3-7e9dd87361ea None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:43.526508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53b244-f46b-4f50-bfa3-7e9dd87361ea None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:43.527260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53b244-f46b-4f50-bfa3-7e9dd87361ea None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:43.530717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 479/942] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:43.542694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:43.543762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:43.544540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:43.578467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:43.580976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:43.640981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41e038-1e18-4b20-8c69-45eb01db827f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:43.704083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:43.705406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:43.705864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:43.706332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:43.719965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:43.725490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:43.725734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 480/943] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:43.755394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:43.757881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:43.759887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 464/944] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:43.765029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:43.773759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:43.781098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:43.838555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:43.934046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58479f-e65d-4757-9916-73982da49b50 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:43.968815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe615-fc3d-49e3-b9a6-082d3f8f18e1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:43.999807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.000955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.001507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.002051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.003420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.006341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.011914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING keystone.server.flask.common [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 16:09:44.017073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.018139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.018555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.018952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.020148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:44.020660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.022394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c930ef3-387e-4ddb-866f-698e4ccc123e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.038749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 465/945] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:44.043833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24ea9-ac7c-4ad7-8c6b-9804d80975b2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.044263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24ea9-ac7c-4ad7-8c6b-9804d80975b2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.044808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24ea9-ac7c-4ad7-8c6b-9804d80975b2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.047272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.sql.core [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 16:09:44.047272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 16:09:44.047272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [parameters: {'id': 'd336418f517041439072167f73b0b18f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 16:09:44.047272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80225) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 16:09:44.053398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 466/946] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:44.054344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be24ea9-ac7c-4ad7-8c6b-9804d80975b2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-49ab607a-bdcc-4f03-b5b6-14e1304f7671 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d336418f517041439072167f73b0b18f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d336418f517041439072167f73b0b18f. Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:09:44.056795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 16:09:44.058365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application yield resource Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise self.value Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application session.commit() Jul 25 16:09:44.060923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.flush() Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application compat.raise_( Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 16:09:44.063106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 16:09:44.065224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:09:44.067050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [parameters: {'id': 'd336418f517041439072167f73b0b18f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.069613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:44.072171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d336418f517041439072167f73b0b18f. Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 481/947] 10.222.0.66 () {60 vars in 1140 bytes} [Mon Jul 25 16:09:43 2022] POST /identity/v3/projects => generated 262 bytes in 305 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.075036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.077103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.101493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:44.103538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:44.194409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 482/948] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:44.194963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:44.203284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:44.243752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:44.264349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be24ea9-ac7c-4ad7-8c6b-9804d80975b2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:44.326925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.328215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.328631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.329059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/6b4413d1949d4f88935b5f3c39c0ac4b/roles/d2864cce36cf4108a69a08feb8529ea6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.333160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=6b4413d1949d4f88935b5f3c39c0ac4b, role_id=d2864cce36cf4108a69a08feb8529ea6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.335589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.357315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 467/949] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:09:44 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/6b4413d1949d4f88935b5f3c39c0ac4b/roles/d2864cce36cf4108a69a08feb8529ea6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:09:44.358771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:44.400395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f8c4-372b-4442-8332-fe842d2ca74e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:44.468336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.474258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.474767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.475251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.487927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.491633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:44.517215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:44.518883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:44.520674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:44.525911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 483/950] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:44.529002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:44.604345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85518-ff35-4326-8354-a68e3b037e18 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:44.664067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:44.685950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.687356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.687834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.688284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.689372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.690941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:44.714765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:44.717063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:44.718908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 468/951] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:44.723763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:44.733393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:44.767740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2aaec-59cc-4378-acb6-707451891730 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:44.844520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:44.845693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:44.846181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:44.846658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:44.848095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:44.848674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:44.851872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:44.905934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 484/952] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:44.957664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5eb19-097d-4fd9-a7a4-6ad9cab7d189 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:45.031899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:45.033304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:45.033990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:45.034564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:45.036113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:45.036806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:45.039672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:45.050000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 469/953] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:45.056358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:45.271702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadce1f7-674c-494e-802b-70e7cbdecefe None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:45.335814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:45.336979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:45.337583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:45.338059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:45.339359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:45.339893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:45.342556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d1df1-9b43-4705-a1fa-d47758da85db None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:45.352914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 485/954] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:45.359109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:45.505232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:45.571275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb488d-e78e-4c1c-9e61-f0976daa2113 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:45.572056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb488d-e78e-4c1c-9e61-f0976daa2113 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:45.572353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb488d-e78e-4c1c-9e61-f0976daa2113 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:45.575434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 486/955] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:45.584804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:45.585529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1bb6b-4d53-4675-b64e-0e48c8cdcf25 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:45.586370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:45.587174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:45.623795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:45.626753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:45.631669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:45.660025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:45.661274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:45.661837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:45.662339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:45.663831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:45.667279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:45.713382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 470/956] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:45.809975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 487/957] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:45.856786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:45.995312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433386-7e05-4562-a85c-04ff4e1cd9fe None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:46.009249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:46.059860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:46.060742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:46.061132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:46.061515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:46.062205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:46.063975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:46.078203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:46.080219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:46.081979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 471/958] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:46.094954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:46.305887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d3037-cbad-4606-a005-5cc4ef3e5035 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:46.376649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:46.378473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:46.379248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:46.380081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:46.382233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] RBAC: Adding query filter params (name=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:46.384246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:46.387407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c5901-b947-4e8e-a08f-12f1482f32de None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:46.394901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:46.398694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 488/959] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:46.408479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:46.627650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88ed24-1ed7-43c2-85a1-dd201d69638a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:46.699122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:46.700313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:46.700691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:46.701178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:46.702568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:46.703124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:46.705889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32417cb-728a-4cf9-aa6a-357bfee597d4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:46.715775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 472/960] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:47.496858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5f62-b7c8-4d27-b872-6207ac3f76ca None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:47.497692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5f62-b7c8-4d27-b872-6207ac3f76ca None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:47.497692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5f62-b7c8-4d27-b872-6207ac3f76ca None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:47.499495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 489/961] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:47.510917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:47.511934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:47.512715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:47.545120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:47.547849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:47.607647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:47.648224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a629a0-ba74-4e14-bf3c-963844ef1567 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:47.648689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a629a0-ba74-4e14-bf3c-963844ef1567 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:47.649142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a629a0-ba74-4e14-bf3c-963844ef1567 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:47.652021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 490/962] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:47.660256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:47.660579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:47.660821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:47.687915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:47.689257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:47.692144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 473/963] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:47.729835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:47.805748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 491/964] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:47.835099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:47.923198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26ae5e-f230-4abb-b0cf-7c89aa990431 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:47.977487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:47.978804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:47.979315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:47.979809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:47.980913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:47.983344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:47.988307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING keystone.server.flask.common [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 16:09:48.005363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.sql.core [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 16:09:48.005363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 16:09:48.005363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [parameters: {'user_id': 'c5302d26db6f45cd88852f07a9794261', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 16:09:48.005363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80226) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 25 16:09:48.028852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:09:48.029978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application yield resource Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise self.value Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application session.commit() Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 16:09:48.031238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.flush() Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application compat.raise_( Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 16:09:48.032295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 16:09:48.033270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 16:09:48.034291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c5302d26db6f45cd88852f07a9794261', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:48.035380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 16:09:48.036361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 16:09:48.039727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.041552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 474/965] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 25 16:09:47 2022] POST /identity/v3/users => generated 160 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:48.049528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.075269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee3f3-da73-4920-8cc1-c8eba497d08a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.094150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:48.145172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:48.165961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.169407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.170006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.170548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.197603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:09:48.206762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:48.208673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:09:48.210961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:48.213662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 492/966] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:09:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:48.214714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.253776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280c4f-adc6-49eb-96f4-8862265e0ebf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.321153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.322516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.323696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.323696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.335704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:48.337599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-13e2081b-1af1-4e79-b1e3-8d5f5cabad22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:48.356965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:48.358905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 475/967] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:48.360757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1187d9-c2ea-414c-8c69-ba04095bb9ce None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.362110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1187d9-c2ea-414c-8c69-ba04095bb9ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.362110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1187d9-c2ea-414c-8c69-ba04095bb9ce None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.367239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac16f83-e44f-44f3-a678-afda9222b6ba None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.367722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 476/968] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:48.369657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1187d9-c2ea-414c-8c69-ba04095bb9ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.415272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.416047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.416333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.416607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.417413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:48.417758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:48.419402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726c68c-23c3-4e15-9761-94c72bf1781c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:48.430615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 493/969] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:48.435930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.436420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.436936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.470962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:48.473885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:48.495497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:48.513421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1187d9-c2ea-414c-8c69-ba04095bb9ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.557894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.558757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.559073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.559319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.560214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:48.560480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:48.562326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:48.566735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:48.605465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 494/970] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:48.605465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.621389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 477/971] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:48.641942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.783441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4c1c0-ed00-4351-b72e-cbc3e92ef0ec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.839760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.839760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.839760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.840181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d51d-88a1-48bc-88cc-e00665da9dff None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:48.840733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.853840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:48.855551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:48.884523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:48.890278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:48.891168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:48.891168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:48.891168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:48.891168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:48.891168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 495/972] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:48.897875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.923122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:48.924298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:48.924732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:48.925136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:48.927566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:48.930438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:48.951618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:48.952729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:48.953690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:48.956118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 478/973] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:48.963394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:48.976049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:48.985263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:49.085610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c218b9-4d9b-4a86-be80-6ac67cb73f4b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.126078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee76a3-9a76-4319-970a-daf9d2d3e765 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.140614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.141773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.142358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.142842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.144188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:49.144720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.147394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.170327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 496/974] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:49.172277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.173449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.173769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.174054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.174819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:49.175137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.176693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.176872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.191001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 479/975] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:49.193070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.308245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:49.324790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:49.341058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f5253-d2ec-4858-8fa3-0c59e282184b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.344556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c165b-8ebb-4c95-be8c-ee95c42376e6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.389441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.390412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.390412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.390672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.391400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:49.391713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.393148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.405244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 480/976] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:49.407081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.410148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.411539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.412024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.412590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.423217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.425045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:09:49.455292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:49.456730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:49.458995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 497/977] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:49.466692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.598420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677407e4-bdd2-4b09-87e0-6c5484bde56b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.629778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64092c-d97e-41eb-9839-f0bb4b3b8726 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.642006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.642843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.643140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.643418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.644249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] RBAC: Adding query filter params (interface=admin, service_id=d4c65c7f43774a8792637c7e1a4723f7, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:49.644565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.646127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c46b2-b5e4-41a4-9cca-ba97f8c9e71f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.652135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 481/978] 10.222.0.66 () {56 vars in 1249 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/endpoints?service_id=d4c65c7f43774a8792637c7e1a4723f7&interface=admin®ion_id=RegionOne => generated 193 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:49.653584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076aa90-534b-4aa9-b74e-7b094ccb20f9 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.653957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076aa90-534b-4aa9-b74e-7b094ccb20f9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.654236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076aa90-534b-4aa9-b74e-7b094ccb20f9 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.655614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 482/979] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:49.662289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.662746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.663241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.672332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:49.675665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.676364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.681656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.681656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.681656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:49.681656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.681656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.688034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:49.690104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:49.703532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 498/980] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:49.704382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.719966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:49.794735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 483/981] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:49.796733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.838284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b5e0-69a6-4d9c-b6cb-a3d6a3611db8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:49.893976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:49.894866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:49.895357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:49.895810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:49.896977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:49.899389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:49.907150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 499/982] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:09:49.908556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:49.954360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a20b0e-e218-4067-b2d9-876d47518303 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:50.000364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:50.009237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:50.010116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:50.010410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:50.010791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:50.034421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:50.079762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5e3d8-a0e1-4846-85a1-6ab4605d7a68 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:50.119878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:50.121462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:50.150526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:50.151296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:50.151621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:50.151870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:50.152934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:50.154858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:50.180906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 500/983] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:49 2022] POST /identity/v3/services => generated 242 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:50.300333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 484/984] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:09:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:09:50.310074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:50.514403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96145024-62a2-4e21-8a2e-7a745328af52 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:50.568351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:50.578275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:50.579322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:50.579619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:50.579902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:50.597692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:50.599292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:09:50.623653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:50.625112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:50.626623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:09:50.626623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:09:50.626623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:09:50.626623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:50.626623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 501/985] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:50.634933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:50.702009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:50.830340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f4f39-9ba2-4e27-b3a0-005e3e5191c3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:50.904743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:50.907155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:50.908038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:50.908813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:50.910761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:50.911614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:50.914573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:50.948664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 485/986] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:50.950217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:51.139919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70edbb-8edf-4302-a309-dfaaa3aac591 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:51.207567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:51.208975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:51.209491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:51.209976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:51.223339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:51.226170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:09:51.254830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:51.255980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:09:51.257406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:51.261838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 502/987] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:51.266390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:51.444938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:51.461118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e654-76be-40c9-9ba9-eb63c5344191 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:51.528408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:51.529592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:51.530075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:51.530569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:51.532044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:51.532634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:51.535397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1d013-5bef-4db0-9fb6-845330afabd0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:51.560467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 486/988] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:51.570067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:51.656102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:51.768686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184cd49-ee2e-4051-9fce-70d2abe3e2e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:51.836001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:51.837140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:51.837648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:51.838160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:51.839887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] RBAC: Adding query filter params (role.id=d2864cce36cf4108a69a08feb8529ea6, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=6b4413d1949d4f88935b5f3c39c0ac4b) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:51.840359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:51.843155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d55ed-4d14-49e2-8759-87a2ecb1f31d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:51.856811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 503/989] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:09:51 2022] GET /identity/v3/role_assignments?user.id=6b4413d1949d4f88935b5f3c39c0ac4b&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=d2864cce36cf4108a69a08feb8529ea6 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:52.110159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfc7a-f6c7-4705-89ee-ab42eca2fb5e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.110562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfc7a-f6c7-4705-89ee-ab42eca2fb5e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.111023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfc7a-f6c7-4705-89ee-ab42eca2fb5e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.114224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 487/990] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:52.121397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.122094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.122247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.151135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:52.153097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:52.291065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 504/991] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:52.330344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbfc7a-f6c7-4705-89ee-ab42eca2fb5e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:52.540909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbfc7a-f6c7-4705-89ee-ab42eca2fb5e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:52.608320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:52.609828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.610322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.610808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.612174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:52.615296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:52.641593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:52.643449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:09:52.645009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 488/992] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 25 16:09:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:52.652163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:52.710910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:52.844327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec789-1d41-4aa0-9367-15f335603a96 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:52.914082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:52.915261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.915709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.916200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.917625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:52.918178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:52.920857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086751-70e9-4522-8015-3a8c59223ef9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:52.931125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 505/993] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:52.940642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:52.945260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fc2f-88f4-408f-950e-cd3aecdb7c6a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.945873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fc2f-88f4-408f-950e-cd3aecdb7c6a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.946393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fc2f-88f4-408f-950e-cd3aecdb7c6a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.949389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 506/994] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:52.958584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:52.959093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:52.959607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:52.976276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:52.990916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:52.993073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:53.123849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 507/995] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:53.142155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3937ea-e40f-42b1-b48f-984a1e7059c6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:53.162320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:53.209504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:53.210446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:53.210894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:53.211341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:53.212570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:53.213067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:53.215550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:53.219341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:53.224413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 489/996] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 25 16:09:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:53.231745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:53.350120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9ccd-2c36-47e7-a53f-ccc278354683 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:53.355318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:53.394637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:53.395534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d6d05-bfc6-4d14-b9da-66f2c0a411ab None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:53.395773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:53.396065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:53.396345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:53.397036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:53.398812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:53.415928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:53.417241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:53.423496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 508/997] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:53.424398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:53.461706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:53.462815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:53.463297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:53.463792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:53.465175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:53.467829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:53.496636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 490/998] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:09:53.591585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:53.605632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb0ff-abce-4c17-9648-bded2cc4768c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:53.684618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:53.685829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:53.686318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:53.686819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:53.688228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:53.688753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:53.691558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:53.701580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 509/999] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:09:53.708222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:53.887064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb1a9-0a01-4e4a-8f9c-246064612e04 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:53.935164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:53.935867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:53.936142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:53.936422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:53.937161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:53.937557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:53.938986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:53.951446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 491/1000] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:53.953155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:54.159393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b92a3-383e-4b28-8fad-d2ee723bfbfa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:54.224655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:54.225830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:54.226351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:54.226871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:54.228435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] RBAC: Adding query filter params (interface=public, service_id=8a637a83a63b41c5bb2b8bb15f204b7f, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:54.228958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:54.231536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0989e-1242-4b89-944f-ce44b6f220bc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:54.246096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 510/1001] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:09:53 2022] GET /identity/v3/endpoints?service_id=8a637a83a63b41c5bb2b8bb15f204b7f&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:54.247734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:54.252172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:54.381595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a62fa-62e2-4e02-8912-0b465510a966 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:54.427639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:54.428371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:54.428652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:54.428947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:54.429707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:54.431378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152dfee-11e5-4354-97d7-401ea9a6e0c8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:54.453662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 492/1002] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:09:54 2022] GET /identity/v3/services => generated 1727 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:09:54.896737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59bbcf-cd76-45ea-b9eb-b4e05bf4d423 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:54.897269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59bbcf-cd76-45ea-b9eb-b4e05bf4d423 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:54.897807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59bbcf-cd76-45ea-b9eb-b4e05bf4d423 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:54.900509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 511/1003] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:54.909469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:54.910072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:54.910666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:54.942854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:54.944933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:55.076106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:55.087264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 493/1004] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:55.124547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59bbcf-cd76-45ea-b9eb-b4e05bf4d423 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:55.164451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:55.296458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59bbcf-cd76-45ea-b9eb-b4e05bf4d423 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:55.343769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:55.344742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:55.345054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:55.345390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] PATH_INFO: `/v3/services/image` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:55.346328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:55.347827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:55.362287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:55.364400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:55.366282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 512/1005] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 25 16:09:55 2022] GET /identity/v3/services/image => generated 86 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:09:55.377898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:55.547174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a251b4-7d3a-44c7-9511-ab26fb839a11 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:55.594320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:55.595036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:55.595316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:55.595607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:55.596357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] RBAC: Adding query filter params (name=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:55.596669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:55.598778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4d02-7553-4315-aaac-f10cb0bddb2c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:55.613502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 494/1006] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:55.616931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:55.778922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:55.815194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0876fa-029e-4209-b377-dcd5e7764950 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:55.879496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:55.880728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:55.881223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:55.882053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:55.883332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] RBAC: Adding query filter params (type=image) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:55.883870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:55.886564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82854d2-4d58-4a62-9fe6-fa90d941cb5d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:55.901475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 513/1007] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:56.180404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bdb0-5ee3-4f27-95a4-5bd055579bdc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.180790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bdb0-5ee3-4f27-95a4-5bd055579bdc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.181354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bdb0-5ee3-4f27-95a4-5bd055579bdc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.184021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 495/1008] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:56.193647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.194229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.194605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.230734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:56.232337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:56.328824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 514/1009] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:56.355006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887bdb0-5ee3-4f27-95a4-5bd055579bdc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:56.488670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:56.569075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887bdb0-5ee3-4f27-95a4-5bd055579bdc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:56.637652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:56.639127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.639640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.640122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.642591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:56.645560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591179d-76e0-45db-b32e-6731cae0c112 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:56.649160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 496/1010] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:09:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:56.658571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:56.772920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:56.809282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847733cd-171e-4ac9-b0d5-22f3c3536806 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:56.856783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:56.857763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.858193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.858600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.859499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:56.861300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:56.875440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316816-0617-4760-a346-0a86539ed3c7 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.875905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316816-0617-4760-a346-0a86539ed3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.876463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316816-0617-4760-a346-0a86539ed3c7 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.879410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 497/1011] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:09:56.888393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:56.888828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:56.889298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:56.903396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 515/1012] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:56 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:09:56.920340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:56.922618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:57.069956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 498/1013] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:09:57.101506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:57.276395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985cd9-c257-47ec-ab9b-64e45149a87e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:57.327586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:57.344008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:57.345534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:57.346032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:57.346513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:57.347767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:57.350693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:09:57.374381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:09:57.375609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 16:09:57.376480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:09:57.377231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 516/1014] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:09:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:09:57.385216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:57.510351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:57.591060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc8d51-454f-4e29-9b47-54e83f06bc1f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:57.658657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:57.659854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:57.660351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:57.661066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:57.662418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:57.662972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:57.665613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:57.675320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 499/1015] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:57.684198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:57.821510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfbbb6-6395-4e2c-831a-2aead8941ba6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:57.871442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:57.872042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:57.872321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:57.872597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:57.873593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:09:57.873906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:57.875367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b4a91-3cda-494f-a8c7-6d700550179e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:57.883194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 517/1016] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 25 16:09:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:09:57.889247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:58.033387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfd917-abda-4f78-8816-75bdbcc65673 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:58.064895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:58.080299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:58.080994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:58.081285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:58.081604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:58.082395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:58.083858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473e45-114a-4b1f-806e-232b8f03be99 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:58.115747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 500/1017] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:09:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:09:58.549179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec517e6a-7eec-4b38-8e2f-a0ed493cfd95 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:58.550056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec517e6a-7eec-4b38-8e2f-a0ed493cfd95 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:58.550056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec517e6a-7eec-4b38-8e2f-a0ed493cfd95 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:58.552025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 518/1018] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:09:58.558806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c7b38-4010-4de4-8737-ee30a3a29494 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:58.559297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c7b38-4010-4de4-8737-ee30a3a29494 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:58.559567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c7b38-4010-4de4-8737-ee30a3a29494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:58.587737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f71c7b38-4010-4de4-8737-ee30a3a29494 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:09:58.589059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f71c7b38-4010-4de4-8737-ee30a3a29494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:09:58.729423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 501/1019] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:09:58 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:09:58.754166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec517e6a-7eec-4b38-8e2f-a0ed493cfd95 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:09:58.830001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:09:58.895727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec517e6a-7eec-4b38-8e2f-a0ed493cfd95 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:09:58.940890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:09:58.941662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:09:58.942189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:09:58.942501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:09:58.943263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:09:58.944855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4afa85-8d0b-43b1-8118-5541a828e976 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:09:58.965111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 519/1020] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:09:58 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:00.295096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb030-c4dd-4572-9739-7815c248f760 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:00.295733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb030-c4dd-4572-9739-7815c248f760 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:00.295733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb030-c4dd-4572-9739-7815c248f760 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:00.297436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 502/1021] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:00.303473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:00.304101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:00.304101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:00.324935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:00.326252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:00.418247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 520/1022] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:00.443796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb030-c4dd-4572-9739-7815c248f760 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:00.576836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb030-c4dd-4572-9739-7815c248f760 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:00.586210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:00.630631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:00.631400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:00.632287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:00.632287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:00.633798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:00.635362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf5ff1-4768-4248-a2f7-eb65cc1da634 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:00.637479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 503/1023] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:00.643971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:00.702214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:00.777778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972f366-5201-414f-bd2a-ca39f6b9a8e3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:00.822048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:00.822807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:00.823020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:00.823322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:00.824069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:00.825631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e8a0f-9200-483a-8e94-07908eaffb12 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:00.852328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 521/1024] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:10:00 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:01.275385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab76f-ad53-4816-9161-3bc21eb8ece1 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:01.275929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab76f-ad53-4816-9161-3bc21eb8ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:01.276355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab76f-ad53-4816-9161-3bc21eb8ece1 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:01.279278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 504/1025] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:01.288215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:01.288709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:01.289199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:01.322441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:01.324466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:01.461949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 522/1026] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:01.492386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaab76f-ad53-4816-9161-3bc21eb8ece1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:01.659765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaab76f-ad53-4816-9161-3bc21eb8ece1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:01.727436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:01.729464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:01.730336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:01.731088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:01.757751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-10c0ed02-aee5-46f2-9480-6dff3a48645f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:01.775526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:01.777402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:01.784450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 505/1027] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:10:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:01.792850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:01.848021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:01.981703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50969-905a-44c3-b726-1a1927536762 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:01.982327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50969-905a-44c3-b726-1a1927536762 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:01.982825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50969-905a-44c3-b726-1a1927536762 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:01.989393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 506/1028] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:01.995219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c422-a114-4ee4-a687-293f66087040 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:01.996196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c422-a114-4ee4-a687-293f66087040 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:01.996364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1719ae0-e563-4a9c-986a-e883ab8da667 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:01.997176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c422-a114-4ee4-a687-293f66087040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.031624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f907c422-a114-4ee4-a687-293f66087040 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:02.034360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f907c422-a114-4ee4-a687-293f66087040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:02.046669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.047371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.047650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.048043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.048963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:02.049269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.050831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.061211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 523/1029] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:10:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:02.076238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.091277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:02.171009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 507/1030] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:02.213204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c422-a114-4ee4-a687-293f66087040 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.222569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:02.236038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79c517-3bae-4f1f-b03c-60e4df1c4569 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.292019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.293552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.294057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.294646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.316441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.319115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:02.353282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:02.355142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:02.356130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:02.356130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:02.356130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:02.356130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:02.356130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 524/1031] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:02.365887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.412115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c422-a114-4ee4-a687-293f66087040 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.465323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.466199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.466488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.466832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] PATH_INFO: `/v3/services/image` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.467717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.469232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:02.482317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:02.483554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:02.484564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 508/1032] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/services/image => generated 86 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:02.491456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.526097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:02.547256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3156b15-9236-438c-bda7-3840cfca3ba4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.593649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.594360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.594708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.594923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.595971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:02.596289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.597813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.622865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 525/1033] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:02.624178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.627735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:02.636371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573127-07d2-4668-bb97-66e5bab71212 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.696783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.697984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.698473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.698947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.700268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] RBAC: Adding query filter params (name=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:02.700792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.703424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.720468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 509/1034] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:02.722181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.778154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6617b9-945d-47df-892c-0d6f6ba5f74e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.839300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.840516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.840958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.841433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.854781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.857252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:10:02.884066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:02.894215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:02.896767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 526/1035] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:02.900491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:02.900731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:02.917427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579af16-490a-4129-ac31-9214a6cab788 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:02.985544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:02.986710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:02.987212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:02.987969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:02.989301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] RBAC: Adding query filter params (type=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:02.989750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:02.992436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:02.995777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:03.003446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 510/1036] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:03.014177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:03.058528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35da43-3c4f-42aa-b889-67055547241c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.107821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:03.108557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.108847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.109144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.110299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:03.110606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:03.112582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:03.140248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 527/1037] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:10:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:03.150535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:03.213995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66a39c-a6cb-47c7-aafe-049ddcfd53bb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.280982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:03.282188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.282695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.283211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.284692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] RBAC: Adding query filter params (interface=public, service_id=4d35f5eb7c764d8ca45952c129feaf07, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:03.285263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:03.288044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:03.301406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 511/1038] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:10:03 2022] GET /identity/v3/endpoints?service_id=4d35f5eb7c764d8ca45952c129feaf07&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:03.304065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:03.331023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185de2-34b4-4c51-9082-c42618f9796f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.339067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:03.365330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:03.396660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:03.397681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.398156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.398616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.492280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afeafa-6ab3-4d51-99a6-749b58926280 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.535732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:03.537721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:03.547468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:03.548320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.548320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.548639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.549448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:03.550997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927a8c8-7a03-45c4-bcc4-c41b2db5f183 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:03.559847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 512/1039] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:10:03 2022] GET /identity/v3/services => generated 1957 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:10:03.562093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72c7e-53fa-4764-91f6-5024d562bb83 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.562602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72c7e-53fa-4764-91f6-5024d562bb83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.563097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72c7e-53fa-4764-91f6-5024d562bb83 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.565982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 513/1040] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:03.572529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.572993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.573472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.603577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:03.605458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:03.729386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 528/1041] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:03.738257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:03.742610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 514/1042] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:03.772428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:03.782468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:03.800924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:03.921549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3688e-6c5b-41af-9176-7ad02b2187c8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.936171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fed32-0a7e-47db-9e3d-e7169caa1503 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:03.978453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:03.979968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:03.980462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:03.980935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:03.998889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:04.008994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:04.011077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:04.012791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 515/1043] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:04.016834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.017862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.018658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.018658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.020302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.033108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.034750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:04.070055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:04.071873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:04.073730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:04.073730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:04.073730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:04.073730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:04.073730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 529/1044] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:04.082208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.184444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:04.208560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:04.242004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d344d3-e17b-41f1-84b8-71e1a53ae851 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.252035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33556fa-8e4e-4810-bd29-4d2014d29b2e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.292580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.293296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.293615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.293896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.294772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:04.295098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.296550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.304631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.305990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.306564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.307135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.314737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 516/1045] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:04.315078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:04.315078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.315078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.318091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.334685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 530/1046] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:04.340447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.455965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d257-7772-4d9b-b291-c670e0815329 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.508940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283ca15-ef81-428a-9c08-94e9830610ec None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.514838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.516401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.516910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.517452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] PATH_INFO: `/v3/users/32d46fb2bba74164adaff697e1e30f3d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.523718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:04.539210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] RBAC: Authorizing `identity:get_user(user_id=32d46fb2bba74164adaff697e1e30f3d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.540829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.550328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 517/1047] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/users/32d46fb2bba74164adaff697e1e30f3d => generated 285 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:04.551417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:04.553511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.558487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.559411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.559709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.559882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.571320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.572893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:10:04.589547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:04.591323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:04.593105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 531/1048] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:04.598173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.724287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd17de-cbe6-4a9a-ae79-ccc8fa8be09e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.775078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b70a-5a19-45e0-9bbe-f5bd070e9aa4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:04.789663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.790820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.791309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.791787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.825773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:04.826884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:04.827921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:04.828476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:04.829559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:04.830092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.831814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.855927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 532/1049] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:04.866219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:04.886292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:04.896876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:04.898954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:04.971778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:05.052641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360b35-486c-445e-9345-e343afde2b43 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:05.077633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 518/1050] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:05.085252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:05.134571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:05.135708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:05.136176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:05.136660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:05.138323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] RBAC: Adding query filter params (role.id=d2864cce36cf4108a69a08feb8529ea6, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:05.138806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:05.141476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:05.152102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 533/1051] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:04 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=d2864cce36cf4108a69a08feb8529ea6 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:05.245826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:05.261235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdd5e0-09ff-4d0d-9d95-89dad486b6e6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:05.331504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:05.333423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:05.334115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:05.334761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:05.337421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:05.340397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b743f3-3d78-43dd-a210-943b1f86dd2d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:05.343642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 519/1052] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:05 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:05.351335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:05.550408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868ca43-d90a-452d-8039-bb965dc0222e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:05.586006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:05.635410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4b85-7960-492b-98fd-366543347129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:05.636538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4b85-7960-492b-98fd-366543347129 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:05.637007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4b85-7960-492b-98fd-366543347129 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:05.637529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4b85-7960-492b-98fd-366543347129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:05.639099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4b85-7960-492b-98fd-366543347129 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=8d17d8739ac74275bf7c856908a401c8, user.id=32d46fb2bba74164adaff697e1e30f3d) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:05.639627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4b85-7960-492b-98fd-366543347129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:05.642425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4b85-7960-492b-98fd-366543347129 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:05.661395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 534/1053] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:05 2022] GET /identity/v3/role_assignments?user.id=32d46fb2bba74164adaff697e1e30f3d&scope.project.id=8d17d8739ac74275bf7c856908a401c8&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:06.113654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736119-6c8a-4ac4-87e0-317cf49dea38 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:06.114512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736119-6c8a-4ac4-87e0-317cf49dea38 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:06.114650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736119-6c8a-4ac4-87e0-317cf49dea38 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:06.116530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 520/1054] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:06.122940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:06.123373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:06.126017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:06.148185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:06.149673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:06.265638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 535/1055] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:06.287698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736119-6c8a-4ac4-87e0-317cf49dea38 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:06.440789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736119-6c8a-4ac4-87e0-317cf49dea38 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:06.511300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:06.512653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:06.513164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:06.513703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] PATH_INFO: `/v3/services/image` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:06.514977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:06.517592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:06.539101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:06.540490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:06.541805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 521/1056] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 25 16:10:06 2022] GET /identity/v3/services/image => generated 86 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:06.549980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:06.748070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c6b7-bb72-4ed3-8b10-885c13d43bba None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:06.771858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:06.819089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:06.820381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:06.820801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:06.821619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:06.822822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:06.823348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:06.826222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:06.836840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 536/1057] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:06.845793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:06.899229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:07.041001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83ab4-8ae5-4c6e-b4d1-795d994f483a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:07.103669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:07.104831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:07.105328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:07.105854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:07.107235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:07.107780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:07.110452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3377fb4-2d82-49cb-a21b-565214ef1bb6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:07.125057 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 522/1058] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:07.127566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:07.287663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1711301-2a56-4bb3-ab0d-eb79cc431d36 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:07.354018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:07.355081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:07.355578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:07.356168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:07.357592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:07.360305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a787d92-0291-43a4-a479-fabb270b4840 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:07.401377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 537/1059] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:10:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:08.204006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518494b-241c-4e81-a11b-1ad10dcdda8d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.204878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518494b-241c-4e81-a11b-1ad10dcdda8d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.205019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518494b-241c-4e81-a11b-1ad10dcdda8d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.207646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 523/1060] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:08.217373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.218157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.218330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.251767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:08.253974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:08.298988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4817-3198-4688-8832-b5ecb7e36f96 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.299361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4817-3198-4688-8832-b5ecb7e36f96 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.299528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf4817-3198-4688-8832-b5ecb7e36f96 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.301324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 524/1061] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:08.304638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:08.307077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.307380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.307636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.336729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:08.338112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:08.392891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 538/1062] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:08.424454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 525/1063] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:08.431557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:08.449542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:08.485405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:08.589847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f63dd-724e-4a98-96d7-5fe43e8d3510 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:08.632063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:08.633009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.633299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.633610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.636193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48ceca-014a-4f2f-8c65-2ecedc16edfc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:08.646130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:08.655130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:08.656489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:08.657645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 539/1064] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:08.665800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:08.690303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:08.695933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:08.697440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.697923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.698392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.716011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:08.724452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:08.725946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:08.727130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:08.728257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 526/1065] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:08.732705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:08.816009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf50-9597-46a8-a88a-db705a5b1fc2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:08.845416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:08.861404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:08.862105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.862466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.862932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.864195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:08.864530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:08.866109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:08.876778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 540/1066] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:08.883348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:08.885147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9e96-7887-42bb-b142-963fbe9ef3d7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:08.931224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:08.932244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:08.932739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:08.933199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:08.934270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:08.934811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:08.936728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:08.949875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 527/1067] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:08.960172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.001710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:09.033691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541188d-8129-4dc4-9998-40b8801707ca None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.084221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.085583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.085876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.086314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.099972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.101519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.102402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12604898-15d5-4465-85ac-6e29b21bddda None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:09.122056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:09.132544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:09.134622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:09.134622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:09.134622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:09.134622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:09.134622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 541/1068] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:09.137404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.168261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.169793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.171016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.171541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] PATH_INFO: `/v3/users/32d46fb2bba74164adaff697e1e30f3d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.173831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=32d46fb2bba74164adaff697e1e30f3d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.176617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.179953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 528/1069] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:08 2022] GET /identity/v3/users/32d46fb2bba74164adaff697e1e30f3d => generated 285 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:09.188980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.210677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:09.279228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8111ed-d5d2-4937-ba58-0ad1bf08f753 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.331956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:09.341746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.342558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.342936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.343308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.344328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:09.344768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.346797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.373870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 542/1070] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:09.380899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.396794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bf7f-a552-4471-b808-e44e7b724ef4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.472452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.473665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.474063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.474498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.476111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.478454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.480549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 529/1071] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:09 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:09.488453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.563992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bd2c1-76c7-4539-9b02-b68c5147770a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.610720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.611732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.612132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.612500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.621760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.623286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.638465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:10:09.640280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:09.650123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:09.652333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 543/1072] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:10:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:09.656007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:09.679000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1dc8c-3b28-4386-8c74-a8d0dbee7afd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.726565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:09.728620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.729796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.730326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.730825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8/users/32d46fb2bba74164adaff697e1e30f3d/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.733313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d17d8739ac74275bf7c856908a401c8, user_id=32d46fb2bba74164adaff697e1e30f3d, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.735096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.765379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 530/1073] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:10:09 2022] PUT /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8/users/32d46fb2bba74164adaff697e1e30f3d/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:10:09.815319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b3b7e-b4c0-4699-8d2e-7119de79c6c8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:09.863523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:09.864352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:09.864804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:09.865211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:09.866296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:09.866767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:09.868495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:09.897133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 544/1074] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:10:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:09.899443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:10.097128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4485e7-0246-4a31-95b9-71b2069821e5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:10.169294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:10.170975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:10.172178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:10.172865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:10.188512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:10.198801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:10.202991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:10.239693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:10.241319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:10.247946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:10.247946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:10.247946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:10.247946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:10.247946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 531/1075] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:10.250545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:10.466208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d595920-3834-4932-9b70-e382bf32585c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:10.486363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:10.537228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:10.538059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:10.538971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:10.538971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:10.539764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:10.540068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:10.542584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5ce3b-cc59-4d89-bc59-55c485e88049 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:10.558266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 545/1076] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:10.564313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:10.608665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fc6b-26cd-409b-9420-c2efcc2229d1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:10.609114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fc6b-26cd-409b-9420-c2efcc2229d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:10.609582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fc6b-26cd-409b-9420-c2efcc2229d1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:10.612788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 546/1077] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:10.621703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:10.622199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:10.622685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:10.651867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:10.654098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:10.735149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e272d9-f8d4-406a-b1ae-55b31a1d079e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:10.757192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 547/1078] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:10.796647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:10.803956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:10.805299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:10.805820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:10.806301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:10.819222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:10.820873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:10:10.837798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:10.839628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:10.842031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 532/1079] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:10.849253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:10.921405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:10.993153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085d941-b215-4263-8986-c3081b246e08 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:11.010699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:11.018698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ddee5-5833-4bd9-97ed-cddb5f316c13 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:11.042340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:11.043047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:11.043324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:11.043602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:11.044433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:11.044742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:11.046434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:11.069398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 533/1080] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:11.073038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:11.096897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:11.098097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:11.098581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:11.099058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:11.217705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e30655-02fa-49e6-9511-c396b37396b0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:11.222572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:11.225376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:11.233534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:11.281628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:11.283348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:11.283835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:11.284342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/d2864cce36cf4108a69a08feb8529ea6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:11.288888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=d73b8801206a4973a97fda3fb00f55c5, role_id=d2864cce36cf4108a69a08feb8529ea6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:11.291706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:11.318179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 534/1081] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:10:11 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/d73b8801206a4973a97fda3fb00f55c5/roles/d2864cce36cf4108a69a08feb8529ea6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:10:11.415285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:11.424687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 548/1082] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 16:10:11.437264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:11.623740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dadaa-0948-4a2b-89a4-2500e492cf51 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:11.670613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:11.671695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:11.672133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:11.672561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:11.686124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:11.687911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1946cc8b-085e-488a-b378-062c4863378b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:11.709224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:11.711246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:11.713194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:11.713194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:11.713194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:11.713194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:11.714015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 535/1083] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:11.722431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:11.912775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1a8b9-62f0-4b12-8d31-af679bbcaeb1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:11.979028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:11.980169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:11.980777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:11.981131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:11.982560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:11.983085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:11.985736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf7614-9c73-4e58-80c0-81af3fdb318f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:12.017396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 549/1084] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:12.161533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d335b0-665c-446e-aa06-356998a80ff9 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:12.161833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d335b0-665c-446e-aa06-356998a80ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:12.162173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d335b0-665c-446e-aa06-356998a80ff9 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:12.163982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 536/1085] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:12.170080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:12.170355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:12.170661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:12.192720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:12.194004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:12.271284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:12.296195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 550/1086] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:12.329503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d335b0-665c-446e-aa06-356998a80ff9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:12.371137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:12.522682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d335b0-665c-446e-aa06-356998a80ff9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:12.588350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:12.589706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:12.590210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:12.590678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:12.610840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:12.626055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:12.628001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:12.630019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 537/1087] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:12.639059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:12.829624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e49ea-d477-4a76-af44-b1bff6254936 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:12.896388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:12.897561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:12.898048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:12.898516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:12.899913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:12.900465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:12.903225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:12.919802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 551/1088] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:12.927253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:13.044263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:13.068231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f319-191a-4489-b0b3-216f1464b713 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:13.109525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:13.110357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:13.111075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:13.111075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] PATH_INFO: `/v3/users/32d46fb2bba74164adaff697e1e30f3d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:13.112791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] RBAC: Authorizing `identity:get_user(user_id=32d46fb2bba74164adaff697e1e30f3d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:13.114667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:13.116588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 538/1089] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:12 2022] GET /identity/v3/users/32d46fb2bba74164adaff697e1e30f3d => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:13.123595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:13.188172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:13.251575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6dfad-81e4-41b0-a795-83be1bf65597 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:13.293281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:13.294082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:13.294389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:13.294650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:13.422790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:13.424344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:13.486103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:13.549750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 552/1090] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:13.556356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:13.679056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f27f80-8c13-43b9-885b-d31644488f85 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:13.723932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:13.724835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:13.725117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:13.725512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:13.727147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:13.728725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d288df0-4fb2-4052-9a91-da54d4b86b78 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:13.730821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 539/1091] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:13 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:13.737369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:13.864651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66245d3d-06ae-4858-b0ac-e1162ea2ff53 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:13.906687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:13.907369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:13.907653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:13.907930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:13.908789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=8d17d8739ac74275bf7c856908a401c8, user.id=32d46fb2bba74164adaff697e1e30f3d) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:13.909095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:13.910693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390406ae-40bb-4873-aed8-5cbbe92396bc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:13.918045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 553/1092] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:13 2022] GET /identity/v3/role_assignments?user.id=32d46fb2bba74164adaff697e1e30f3d&scope.project.id=8d17d8739ac74275bf7c856908a401c8&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:14.415483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a560b-9fd4-4512-bab8-794e06146afe None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:14.415829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a560b-9fd4-4512-bab8-794e06146afe None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:14.416115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a560b-9fd4-4512-bab8-794e06146afe None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:14.417687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 540/1093] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:14.427479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:14.427935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:14.428302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:14.451143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:14.452395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:14.466185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:14.583150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 554/1094] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:14.610082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a560b-9fd4-4512-bab8-794e06146afe None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:14.646387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:14.758187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a560b-9fd4-4512-bab8-794e06146afe None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:14.804103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:14.805083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:14.805516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:14.805875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:14.820435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:14.829314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:14.831467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:14.833576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 541/1095] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 25 16:10:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:14.839781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:15.010472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299010b-b0e5-40ea-8412-e58b5c152196 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:15.080500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:15.081717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:15.082298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:15.082797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:15.084250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:15.084911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:15.087679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:15.104191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 555/1096] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:10:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:15.111825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:15.232349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:15.259839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ddb3-f579-496f-adbc-da7c250def16 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:15.299928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:15.300651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:15.300971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:15.301245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:15.316042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:15.318978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:15.355007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:15.356058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:15.357018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:15.357018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:15.357018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:15.357018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:15.357536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 542/1097] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:15.363863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:15.373663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:15.519516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445f54a-beff-45ac-8b99-7db4fa9fb7b8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:15.566169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:15.567366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:15.567846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:15.568137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:15.569319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:15.569892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:15.572629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:15.599224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 556/1098] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:15.608651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:15.780900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7eb4-8e7e-48a6-a912-99335759d6e8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:15.819258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:15.827175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:15.828126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:15.828476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:15.828817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:15.839364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:15.841119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:10:15.858750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:15.860630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:15.862615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 543/1099] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 25 16:10:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:15.869110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:15.998524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:16.051183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36410c-116e-4c28-9057-0b7a832e8d67 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:16.099322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:16.099966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.100228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.100481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.101536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:16.101916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:16.103655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122f19-4117-4198-a01c-2b2e6594eff9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:16.127867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 557/1100] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 25 16:10:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:16.135364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:16.213820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d5aa5-6da9-4182-a56a-01226426a4b8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.214286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d5aa5-6da9-4182-a56a-01226426a4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.214721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d5aa5-6da9-4182-a56a-01226426a4b8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.216269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 558/1101] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:16.222443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.222768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.223009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.244978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:16.246377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:16.315263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1414000-7fcc-4b68-a6d2-49e2313300b6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:16.350524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 559/1102] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:16.365885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:16.366710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.367083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.367450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.378262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:16.446190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:16.458837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:16.470491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:16.472929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:16.534943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06393779-2948-4779-85ba-f6f1d7b661de None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:16.591883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:16.592755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.593048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.593324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.594905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:16.596764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:16.599099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 560/1103] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:16.605706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:16.618959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 544/1104] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:16.625773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:16.744245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46a9d-4428-45f8-87a2-81a95146b96b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:16.757062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:16.769734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2e712-8c6e-462c-85f3-c29c58f8c256 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:16.778592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:16.793354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:16.794085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.794380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.794646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.795397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:16.796914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:16.837451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 561/1105] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:10:16 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:16.837709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:16.837709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:16.837709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:16.837709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:16.857373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:16.860256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:16.895522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:16.896973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:16.898461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:10:16.898461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:16.898461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:10:16.898461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:16.898461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 545/1106] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:16.904982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:17.057076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412b8f6-5c89-4939-a1ba-79a8dcb8a7fd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:17.117829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:17.117829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:17.117829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:17.117829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:17.118688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:17.118688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:17.123906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:17.141008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 562/1107] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:17.148176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:17.278087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e5d44-752d-487c-841b-0eee5d30c28a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:17.321038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:17.321936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:17.322230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:17.322514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:17.331592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:17.333262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:10:17.348920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:17.350376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:17.351294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 546/1108] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:10:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:10:17.359658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:17.416708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:17.495712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6e20-97af-47ac-8dd3-cf243152084f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:17.541620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:17.542322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:17.542665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:17.542995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:17.543986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:17.544347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:17.545920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348afe6f-9722-4997-8de6-a812f5c169d6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:17.560919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 563/1109] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:10:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:17.567521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:17.570817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:17.699229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0133e-6b5d-4c9f-a2f9-be8e34ed38b5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:17.741188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:17.742037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:17.742417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:17.742800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:17.743920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] RBAC: Adding query filter params (role.id=d2864cce36cf4108a69a08feb8529ea6, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=d73b8801206a4973a97fda3fb00f55c5) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:17.744366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:17.746076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca8a65-8518-47da-a63b-4d1d800da4ef None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:17.754886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 547/1110] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:17 2022] GET /identity/v3/role_assignments?user.id=d73b8801206a4973a97fda3fb00f55c5&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=d2864cce36cf4108a69a08feb8529ea6 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:19.380145 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd95-4c93-4c3e-acdb-8e2bfffaacb6 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:19.380758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd95-4c93-4c3e-acdb-8e2bfffaacb6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:19.380758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd95-4c93-4c3e-acdb-8e2bfffaacb6 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:19.382669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 564/1111] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:19.388931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:19.389576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:19.389576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:19.410835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:19.412102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:19.485526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:19.495296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 548/1112] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:19.536631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043fd95-4c93-4c3e-acdb-8e2bfffaacb6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:19.699124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:19.708545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043fd95-4c93-4c3e-acdb-8e2bfffaacb6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:19.761139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:19.762271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:19.762606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:19.762941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] PATH_INFO: `/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:19.764924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] RBAC: Authorizing `identity:get_role(role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:19.766538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:19.768128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 565/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:19 2022] GET /identity/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 223 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:19.774977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:19.930685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62a297-7b11-4ac6-8295-01ba41551de8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:19.977665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:19.978571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:19.978898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:19.979223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] PATH_INFO: `/v3/users/961ffe48952c4655983fbf570f724043` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:19.996759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] RBAC: Authorizing `identity:get_user(user_id=961ffe48952c4655983fbf570f724043)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:19.998310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:20.000169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 549/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:19 2022] GET /identity/v3/users/961ffe48952c4655983fbf570f724043 => generated 286 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:20.007201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:20.192074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e50f0d-4936-4011-8e00-fdf3aa16dc93 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:20.235023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:20.236031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:20.236031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:20.236379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:20.279587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:20.324953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:20.326792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:20.450511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 566/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:20.457541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:20.532616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:20.588555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baeb215-2584-4d15-b9eb-bedd8af2ee10 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:20.636590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:20.637675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:20.638233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:20.638542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:20.640260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:20.642657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2b6d0-1f83-45c3-8829-3703d915efe4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:20.645328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 550/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:20 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:20.655638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:20.742800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:20.786057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288456-d6b3-4ea4-9c76-71afcb56b38d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:20.833351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:20.834054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:20.834339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:20.834625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:20.835695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=8d17d8739ac74275bf7c856908a401c8, user.id=961ffe48952c4655983fbf570f724043) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:20.836201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:20.838114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382df7-bdab-4f85-bab1-916016b199cd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:20.845071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 567/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:20 2022] GET /identity/v3/role_assignments?user.id=961ffe48952c4655983fbf570f724043&scope.project.id=8d17d8739ac74275bf7c856908a401c8&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:22.980197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3b66e-6313-453f-87d0-6b46910031ae None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:22.980804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3b66e-6313-453f-87d0-6b46910031ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:22.980804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3b66e-6313-453f-87d0-6b46910031ae None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:22.982393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 551/1118] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:22.989280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:22.989964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:22.989964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:23.010519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:23.011938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:23.091890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 568/1119] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:23.115528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3b66e-6313-453f-87d0-6b46910031ae None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:23.275582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3b66e-6313-453f-87d0-6b46910031ae None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:23.319308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:23.339623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:23.341105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:23.341644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:23.342084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] PATH_INFO: `/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:23.344801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] RBAC: Authorizing `identity:get_role(role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:23.347611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:23.350318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 552/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:23 2022] GET /identity/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 223 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:23.357694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:23.469901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:23.541400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea451-a1c8-4d65-8b57-80d3b82bbb0a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:23.583408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:23.584423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:23.584750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:23.585080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] PATH_INFO: `/v3/users/961ffe48952c4655983fbf570f724043` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:23.586521 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] RBAC: Authorizing `identity:get_user(user_id=961ffe48952c4655983fbf570f724043)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:23.588095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:23.589932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 569/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:23 2022] GET /identity/v3/users/961ffe48952c4655983fbf570f724043 => generated 286 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:23.596793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:23.723852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57616fdf-6c51-4079-8b94-61f6d0ad6ebe None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:23.776579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:23.777667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:23.778091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:23.778461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:23.780009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:23.781892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec5c2-a634-4f7c-b2c3-097ef93ac50c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:23.783928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 553/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:23 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:23.797985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:23.973273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9f7f-98d2-44ae-a7c3-2a520266d800 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:24.007367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:24.018964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:24.020137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:24.020496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:24.021128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8/users/961ffe48952c4655983fbf570f724043/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:24.023491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d17d8739ac74275bf7c856908a401c8, user_id=961ffe48952c4655983fbf570f724043, role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:24.025146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b5de4-f544-4f9c-94d7-c334cbf25218 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:24.042636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 570/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:10:23 2022] PUT /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8/users/961ffe48952c4655983fbf570f724043/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:10:26.168903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30f6b1-736f-4ed7-9939-33755a04d63b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:26.169594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30f6b1-736f-4ed7-9939-33755a04d63b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:26.170036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30f6b1-736f-4ed7-9939-33755a04d63b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:26.172827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 554/1124] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:26.179376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:26.179993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:26.180180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:26.202314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:26.203572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:26.292151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 571/1125] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:26.320264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30f6b1-736f-4ed7-9939-33755a04d63b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:26.436742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:26.526472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30f6b1-736f-4ed7-9939-33755a04d63b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:26.570218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:26.571115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:26.571394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:26.571667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] PATH_INFO: `/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:26.573317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] RBAC: Authorizing `identity:get_role(role_id=1b9b9a1b98ec47e59a28e1e93665e266)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:26.574781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:26.576300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 555/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:26 2022] GET /identity/v3/roles/1b9b9a1b98ec47e59a28e1e93665e266 => generated 223 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:26.583077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:26.707869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870586e-76d1-4689-9d3a-9bd595a4550d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:26.734558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:26.759550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:26.760525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:26.760807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:26.761066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] PATH_INFO: `/v3/users/961ffe48952c4655983fbf570f724043` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:26.762509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] RBAC: Authorizing `identity:get_user(user_id=961ffe48952c4655983fbf570f724043)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:26.764046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:26.765843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 572/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:26 2022] GET /identity/v3/users/961ffe48952c4655983fbf570f724043 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:26.772835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:26.908332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2fb9a-ece3-4943-a8ce-f5fc13f74848 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:26.942202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:26.953530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:26.954277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:26.954604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:26.954915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:27.041739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:27.043124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:27.180700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 556/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:27.187970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:27.346022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710500-5884-40bf-af88-ea4b91ad81a6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:27.391200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:27.392966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:27.393660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:27.394245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] PATH_INFO: `/v3/projects/8d17d8739ac74275bf7c856908a401c8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:27.396897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d17d8739ac74275bf7c856908a401c8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:27.399700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55682-ebe1-47ef-853b-80fd009232f8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:27.403136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 573/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:27 2022] GET /identity/v3/projects/8d17d8739ac74275bf7c856908a401c8 => generated 311 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:27.412178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:27.452229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:27.560607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bcce8-687f-407d-899e-33aa4faaacba None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:27.628105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:27.629174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:27.629719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:27.630013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:27.630880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] RBAC: Adding query filter params (role.id=1b9b9a1b98ec47e59a28e1e93665e266, scope.project.id=8d17d8739ac74275bf7c856908a401c8, user.id=961ffe48952c4655983fbf570f724043) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:27.631194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:27.633052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f910e-66fc-4fa0-bb6e-ea6ecb2b3ce4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:27.640487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 557/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:27 2022] GET /identity/v3/role_assignments?user.id=961ffe48952c4655983fbf570f724043&scope.project.id=8d17d8739ac74275bf7c856908a401c8&role.id=1b9b9a1b98ec47e59a28e1e93665e266 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:29.875816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206aac-3025-4f7f-9ad9-3d4949aa5893 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:29.876436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206aac-3025-4f7f-9ad9-3d4949aa5893 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:29.876436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3206aac-3025-4f7f-9ad9-3d4949aa5893 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:29.877912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 574/1131] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:29.887178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:29.887827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:29.887827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:29.909569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:29.911307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:30.026322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 558/1132] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:30.065982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3206aac-3025-4f7f-9ad9-3d4949aa5893 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:30.163754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:30.217959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3206aac-3025-4f7f-9ad9-3d4949aa5893 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:30.260139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:30.261110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:30.261470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:30.261791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:30.263344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:30.265104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2b2e3-c623-4072-9c66-f8d3cb4bab3c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:30.266896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 575/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:30.278429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:30.305247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:30.396027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7993-ceeb-47e9-85df-0e942e59b092 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:30.436492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:30.437096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:30.437404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:30.437689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:30.438453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:30.439997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af6a58-f506-41e3-a575-399559636487 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:30.473254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 559/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:10:30 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:32.769444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481111-14c9-408c-9d79-b219e9c910e5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:32.770274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481111-14c9-408c-9d79-b219e9c910e5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:32.770274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481111-14c9-408c-9d79-b219e9c910e5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:32.771641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 576/1135] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:32.779287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:32.779867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:32.780271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:32.812419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:32.814562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:32.914221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 560/1136] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:32.940368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93481111-14c9-408c-9d79-b219e9c910e5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:33.079206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93481111-14c9-408c-9d79-b219e9c910e5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:33.113286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:33.131009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:33.132335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:33.132622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:33.132914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:33.134580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:33.136523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c9ad0-9e99-4068-badf-7dac198ba90c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:33.139897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 577/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:10:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:33.148352 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:33.151088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:33.273064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64eda0-e54f-4741-b4d3-04ccd796d445 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:33.317018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:33.318222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:33.318697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:33.319182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:33.320537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:33.323338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611525a0-7374-438a-bcce-5cf3bd2215c9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:33.360100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 561/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:10:33 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:10:35.569552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedb1b1-726c-42fc-a38f-29332faf453d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:35.570387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedb1b1-726c-42fc-a38f-29332faf453d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:35.570579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedb1b1-726c-42fc-a38f-29332faf453d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:35.573282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 578/1139] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:35.579757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:35.580415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:35.580415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:35.601740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:35.603051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:35.702210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:35.728974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 562/1140] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:35.771291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedb1b1-726c-42fc-a38f-29332faf453d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:35.966537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedb1b1-726c-42fc-a38f-29332faf453d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:36.028562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:36.029894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:36.030393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:36.030881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:36.050867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:36.063845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:36.065525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:36.067349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 579/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:36.072178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:36.223331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487ea89-31ba-422a-b9f5-ecdc1c11e775 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:36.273236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:36.273956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:36.274236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:36.274513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:36.275280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:36.275581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:36.276983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:36.288994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 563/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:36.295475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:36.355031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:36.414416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72d3b3-a65c-4aa4-8ae4-2f583a8275a0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:36.455948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:36.461061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:36.461393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:36.461679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] PATH_INFO: `/v3/users/47577bd92e4546159a7f6aa740599ad1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:36.482001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] RBAC: Authorizing `identity:get_user(user_id=47577bd92e4546159a7f6aa740599ad1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:36.484014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:36.486333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 580/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:36 2022] GET /identity/v3/users/47577bd92e4546159a7f6aa740599ad1 => generated 286 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:36.493849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:36.558581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:36.639953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90b9f4-d324-4b47-9f38-929d30738489 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:36.694772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:36.695804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:36.696111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:36.696389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:36.816198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:36.818979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:36.888915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:36.948547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 564/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:36.955834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:37.082825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0edf05-7fa9-49a9-b3e0-5396f34f0b7c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:37.098533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:37.130279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:37.131179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:37.131464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:37.132104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] PATH_INFO: `/v3/projects/4e8e4407b22247e2b685f383f2dd309f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:37.133522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] RBAC: Authorizing `identity:get_project(project_id=4e8e4407b22247e2b685f383f2dd309f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:37.135022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4da935-6d65-4a00-b28c-d8b6b366e1cb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:37.136847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 581/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:36 2022] GET /identity/v3/projects/4e8e4407b22247e2b685f383f2dd309f => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:37.143377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:37.258707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e938b0-8312-469e-8c8e-fdbc20d9b14b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:37.319363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:37.320567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:37.321073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:37.321625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:37.323343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=4e8e4407b22247e2b685f383f2dd309f, user.id=47577bd92e4546159a7f6aa740599ad1) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:37.323888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:37.326727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebe308-44f9-4301-a3e2-b2e1d575183a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:37.338701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 565/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:37 2022] GET /identity/v3/role_assignments?user.id=47577bd92e4546159a7f6aa740599ad1&scope.project.id=4e8e4407b22247e2b685f383f2dd309f&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:39.761995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febedf85-9362-465f-86c6-4a34afa8ed47 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:39.762849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febedf85-9362-465f-86c6-4a34afa8ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:39.762849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febedf85-9362-465f-86c6-4a34afa8ed47 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:39.764241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 582/1147] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:39.770825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:39.772437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:39.772437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:39.794090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:39.795402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:39.808048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:39.923253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 566/1148] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:39.963298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febedf85-9362-465f-86c6-4a34afa8ed47 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:40.146650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febedf85-9362-465f-86c6-4a34afa8ed47 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:40.207230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:40.208661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:40.209155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:40.209694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:40.231524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:40.247011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:40.248623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:40.249842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 583/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:40.259764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:40.387007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3e929-3063-4967-94af-3153e09fc6b8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:40.430470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:40.431323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:40.431609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:40.431890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:40.432722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:40.433035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:40.435083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:40.446655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 567/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:40.457059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:40.525730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:40.657974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7c38-0254-44dc-845e-12c24f679a82 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:40.702090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:40.703099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:40.703457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:40.703782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] PATH_INFO: `/v3/users/47577bd92e4546159a7f6aa740599ad1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:40.705231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] RBAC: Authorizing `identity:get_user(user_id=47577bd92e4546159a7f6aa740599ad1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:40.709431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:40.711415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 584/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:40 2022] GET /identity/v3/users/47577bd92e4546159a7f6aa740599ad1 => generated 286 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:40.718704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:40.734550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:40.902698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9596b-3cd9-4291-b64d-92da679e415a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:40.962351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:40.963609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:40.963903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:40.964248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] PATH_INFO: `/v3/projects/4e8e4407b22247e2b685f383f2dd309f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:40.965852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8e4407b22247e2b685f383f2dd309f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:40.968084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f5e68-0deb-4e6e-835a-98b9c51d4a75 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:40.970214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 568/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:40 2022] GET /identity/v3/projects/4e8e4407b22247e2b685f383f2dd309f => generated 311 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:40.977098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:41.107853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b73f-c026-439d-a6cf-4e0df68eb5a6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:41.112653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:41.158264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:41.159271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:41.159622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:41.160039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] PATH_INFO: `/v3/projects/4e8e4407b22247e2b685f383f2dd309f/users/47577bd92e4546159a7f6aa740599ad1/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:41.162620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8e4407b22247e2b685f383f2dd309f, user_id=47577bd92e4546159a7f6aa740599ad1, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:41.164189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a0afd-3bc9-46f8-8b20-e69d66fa7714 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:41.179903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 585/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:10:40 2022] PUT /identity/v3/projects/4e8e4407b22247e2b685f383f2dd309f/users/47577bd92e4546159a7f6aa740599ad1/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:10:43.279132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450273d9-4082-4a84-b088-615ae3ae1125 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:43.279740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450273d9-4082-4a84-b088-615ae3ae1125 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:43.280013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450273d9-4082-4a84-b088-615ae3ae1125 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:43.282530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 569/1154] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:43.291847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:43.292457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:43.292794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:43.325541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:43.327753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:43.431405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 586/1155] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:43.464459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450273d9-4082-4a84-b088-615ae3ae1125 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:43.569188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:43.604122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450273d9-4082-4a84-b088-615ae3ae1125 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:43.642874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:43.643697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:43.644017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:43.644295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:43.657555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:43.665419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:43.666838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:43.668754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 570/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:43.673590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:43.789877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:43.798928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11da31-42b4-4f2c-9375-eb0e7136a13f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:43.849227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:43.850043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:43.850391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:43.850751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:43.851570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:43.851943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:43.854103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:43.867957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 587/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:43.874644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:44.003336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c507f-202e-42b6-a8f8-85b06aeb5e5c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:44.050668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:44.051584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:44.051862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:44.052135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] PATH_INFO: `/v3/users/47577bd92e4546159a7f6aa740599ad1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:44.053528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] RBAC: Authorizing `identity:get_user(user_id=47577bd92e4546159a7f6aa740599ad1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:44.055034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:44.056813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 571/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:43 2022] GET /identity/v3/users/47577bd92e4546159a7f6aa740599ad1 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:44.063373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:44.194992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c417ef-2cf0-48a0-b596-96ec3fcc74a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:44.264475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:44.265397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:44.265738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:44.266023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:44.314477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:44.405152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:44.407557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:44.581764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 588/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 16:10:44.588231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:44.591547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:44.753379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe540e-5079-4933-b23b-cd60d0fd6f6b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:44.798862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:44.800437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:44.800946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:44.801464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] PATH_INFO: `/v3/projects/4e8e4407b22247e2b685f383f2dd309f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:44.803800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8e4407b22247e2b685f383f2dd309f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:44.806621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590022-b483-4d45-bc15-6842ac54c8d4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:44.809680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 572/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:10:44 2022] GET /identity/v3/projects/4e8e4407b22247e2b685f383f2dd309f => generated 311 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:44.818894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:44.911120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:44.979258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c02229-2947-4b89-96f0-81e6b68ae12d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:45.030353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:45.031495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:45.031959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:45.032424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:45.034052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=4e8e4407b22247e2b685f383f2dd309f, user.id=47577bd92e4546159a7f6aa740599ad1) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:45.034593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:45.037465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b0f17-94ca-4e7d-9c85-9de0f8f6d39f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:45.048315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 589/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:10:44 2022] GET /identity/v3/role_assignments?user.id=47577bd92e4546159a7f6aa740599ad1&scope.project.id=4e8e4407b22247e2b685f383f2dd309f&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:47.259532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c304-71f9-46ad-b4b5-4a5534cce6f3 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:47.260469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c304-71f9-46ad-b4b5-4a5534cce6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:47.260469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c304-71f9-46ad-b4b5-4a5534cce6f3 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:47.261725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 573/1162] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:47.268156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:47.268772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:47.268772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:47.290309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:47.291691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:47.393479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 590/1163] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:47.416636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329c304-71f9-46ad-b4b5-4a5534cce6f3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:47.495456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:47.537776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329c304-71f9-46ad-b4b5-4a5534cce6f3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:47.578853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:47.579527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:47.579809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:47.580085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:47.678181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:47.680137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:47.771070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:47.810443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 574/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:10:47.817062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:47.883864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:47.951336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66e222-e933-42ed-a37f-2bfcfef128db None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:47.990284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:47.991169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:47.991488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:47.991801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:47.999359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:48.000864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9f0feae2-3016-4afe-930b-1c1e9685f606 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:48.016222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:48.017619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:10:48.018689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 591/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:10:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:48.025489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:48.146162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288b74-193a-4c69-90d6-f66211512eb9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:48.186490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:48.187178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:48.187461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:48.187745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:48.188497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:48.188812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:48.190313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75ca37-d9f6-4e0f-924e-d8a9e2ac7553 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:48.199066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 575/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 25 16:10:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:50.534108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422214-0f17-4f8c-a6c2-7f7745a1c4eb None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:50.535055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422214-0f17-4f8c-a6c2-7f7745a1c4eb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:50.535055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422214-0f17-4f8c-a6c2-7f7745a1c4eb None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:50.536713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 592/1167] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:50.543128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860725-87f1-41ba-b377-59b50cb91a87 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:50.543779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860725-87f1-41ba-b377-59b50cb91a87 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:50.543779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860725-87f1-41ba-b377-59b50cb91a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:50.565760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-72860725-87f1-41ba-b377-59b50cb91a87 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:50.567032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-72860725-87f1-41ba-b377-59b50cb91a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:50.613892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:50.689629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 576/1168] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:50.715070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422214-0f17-4f8c-a6c2-7f7745a1c4eb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:50.869159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422214-0f17-4f8c-a6c2-7f7745a1c4eb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:50.883618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:50.911474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:50.912166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:50.912526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:50.912807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:50.913614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:50.915053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7db6-7d9f-4af9-945c-a72127415bdc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:50.935298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 593/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Mon Jul 25 16:10:50 2022] POST /identity/v3/domains => generated 267 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:53.083980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3de2e-3b9d-4913-81c5-03a7b00417a2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:53.084719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3de2e-3b9d-4913-81c5-03a7b00417a2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:53.084719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3de2e-3b9d-4913-81c5-03a7b00417a2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:53.086183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 577/1170] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:10:53.092219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:53.092971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:53.092971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:53.112658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:53.113941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:53.223559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 594/1171] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:53.250145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3de2e-3b9d-4913-81c5-03a7b00417a2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:53.375213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3de2e-3b9d-4913-81c5-03a7b00417a2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:53.414649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:53.415543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:53.415837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:53.416098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] PATH_INFO: `/v3/domains/8d67fae7ea14433e869ac677a026654b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:53.428907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8d67fae7ea14433e869ac677a026654b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:53.430462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6861b-85a0-4e52-b6b0-f7fc49f68c39 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:53.432296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 578/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Mon Jul 25 16:10:53 2022] GET /identity/v3/domains/8d67fae7ea14433e869ac677a026654b => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:53.441997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:53.566224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:53.570347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b3b9-705d-46c8-b137-d6b2e889a5ac None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:53.630832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:53.632256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:53.632759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:53.633260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:53.634652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:53.637431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3486c03-a763-4270-91dc-fa250ba11494 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:53.677128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 595/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:10:53 2022] POST /identity/v3/projects => generated 361 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:56.028900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008005bf-dde8-41a7-895d-b361270822fa None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:56.029575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008005bf-dde8-41a7-895d-b361270822fa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:56.029922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008005bf-dde8-41a7-895d-b361270822fa None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:56.032617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 579/1174] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:56.039006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:56.039640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:56.039640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:56.059833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:56.061098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:56.152928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 596/1175] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:10:56.189011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008005bf-dde8-41a7-895d-b361270822fa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:56.242719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:56.407131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008005bf-dde8-41a7-895d-b361270822fa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:56.465190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:56.466848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:56.467336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:56.467834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] PATH_INFO: `/v3/domains/8d67fae7ea14433e869ac677a026654b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:56.470297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=8d67fae7ea14433e869ac677a026654b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:56.473322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965cd66-76d9-4794-8594-854d5358b90b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:56.476696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 580/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Mon Jul 25 16:10:56 2022] GET /identity/v3/domains/8d67fae7ea14433e869ac677a026654b => generated 267 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:10:56.486665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:56.638858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:56.667289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38176f39-fc1a-4c86-9c2c-8d323a6d9db6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:56.717148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:56.717949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:56.718284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:56.718611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:56.719412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:56.720994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a3c1d-3769-4d65-b7ac-22371cf1b042 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:56.751494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 597/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:10:56 2022] POST /identity/v3/users => generated 311 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:10:59.257570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074e6e-9577-4665-9af4-baa869799514 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:59.258459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074e6e-9577-4665-9af4-baa869799514 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:59.258631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074e6e-9577-4665-9af4-baa869799514 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:59.261166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 581/1178] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:10:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:10:59.268803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:59.269444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:59.269826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:59.295872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:10:59.297177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:10:59.379090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 598/1179] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:10:59.402970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad074e6e-9577-4665-9af4-baa869799514 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:59.527034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:59.539305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad074e6e-9577-4665-9af4-baa869799514 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:59.578958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:59.579820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:59.580103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:59.580379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:59.594203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:10:59.602346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:10:59.604164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:10:59.605841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 582/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:10:59.610662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:59.690370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:10:59.731709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284ce39-59a6-4ca2-9917-bb3de497efd2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:59.776449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:59.777220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:59.777480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:59.777748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:59.778555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:10:59.778869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:59.780352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:59.791246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 599/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:10:59.798053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:10:59.923752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06da6b-3ab1-4d25-9603-0967f3bf9c79 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:10:59.965481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:10:59.966374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:10:59.966647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:10:59.966920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] PATH_INFO: `/v3/users/e237e18bd09e47d897635c34512844cd` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:10:59.986534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] RBAC: Authorizing `identity:get_user(user_id=e237e18bd09e47d897635c34512844cd)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:10:59.989777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:10:59.993161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 583/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:10:59 2022] GET /identity/v3/users/e237e18bd09e47d897635c34512844cd => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:00.000335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:00.162833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78cc0c-bb7f-4951-9c98-1ba565b8b052 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:00.203286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:00.212814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:00.213558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:00.213847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:00.214182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:00.305543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:00.307044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:00.423393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 600/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:10:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 16:11:00.430317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:00.447593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:00.555108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb454337-ac0b-4f5c-a9a9-f8055535daeb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:00.597451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:00.598417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:00.598758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:00.599076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] PATH_INFO: `/v3/projects/705e0f35646f490394d9448a8dad03e2` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:00.600496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] RBAC: Authorizing `identity:get_project(project_id=705e0f35646f490394d9448a8dad03e2)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:00.602122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6f454-da1b-4c7c-9a7a-b0eae07cfc82 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:00.604060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 584/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:11:00 2022] GET /identity/v3/projects/705e0f35646f490394d9448a8dad03e2 => generated 361 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:00.611383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:00.658319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:00.750607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a2acf-d2c9-4196-aced-fae4889822ff None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:00.792571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:00.793448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:00.793807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:00.794119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:00.795078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=705e0f35646f490394d9448a8dad03e2, user.id=e237e18bd09e47d897635c34512844cd) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:11:00.795415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:00.797031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f820b6a-6c41-430f-ae28-4bc3398c6f63 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:00.804188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 601/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:11:00 2022] GET /identity/v3/role_assignments?user.id=e237e18bd09e47d897635c34512844cd&scope.project.id=705e0f35646f490394d9448a8dad03e2&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:02.961956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ed65-f45b-4ad9-9f8f-a66273fc0a5c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:02.962867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ed65-f45b-4ad9-9f8f-a66273fc0a5c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:02.962867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ed65-f45b-4ad9-9f8f-a66273fc0a5c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:02.963937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 585/1186] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:11:02.971311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:02.972088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:02.972319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:03.005225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:11:03.007776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:11:03.148063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 602/1187] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:11:03.175013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740ed65-f45b-4ad9-9f8f-a66273fc0a5c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:03.279637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:03.306329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740ed65-f45b-4ad9-9f8f-a66273fc0a5c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:03.345373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:03.346307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:03.346775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:03.347043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:03.359321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:11:03.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:11:03.368471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:11:03.370120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 586/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:11:03.376129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:03.412903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:03.517621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e87a2e-1cf4-4282-a281-a8180ead9907 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:03.561944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:03.562588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:03.562910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:03.563192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:03.564044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:11:03.564399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:03.565977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:03.577856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 603/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:03.583778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:03.733188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3ce6-aaf9-4208-a8b0-bc981ff67cc0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:03.776271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:03.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:03.777530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:03.777816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] PATH_INFO: `/v3/users/e237e18bd09e47d897635c34512844cd` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:03.779320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=e237e18bd09e47d897635c34512844cd)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:03.780936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:03.782936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 587/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:11:03 2022] GET /identity/v3/users/e237e18bd09e47d897635c34512844cd => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:11:03.789626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:03.902938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:03.924481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f767a-dcec-4171-8c35-5379c16f3972 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:03.973209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:03.974448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:03.974759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:03.975049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] PATH_INFO: `/v3/projects/705e0f35646f490394d9448a8dad03e2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:03.976657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] RBAC: Authorizing `identity:get_project(project_id=705e0f35646f490394d9448a8dad03e2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:03.978517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf9a7-9233-479e-a115-f5dd735178a9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:03.980696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 604/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:11:03 2022] GET /identity/v3/projects/705e0f35646f490394d9448a8dad03e2 => generated 361 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:03.987994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:03.999228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:04.128464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769938f-7de9-4381-87cf-e82171aed1c0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:04.172386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:04.173590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:04.173987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:04.174361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] PATH_INFO: `/v3/projects/705e0f35646f490394d9448a8dad03e2/users/e237e18bd09e47d897635c34512844cd/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:04.177033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] RBAC: Authorizing `identity:create_grant(project_id=705e0f35646f490394d9448a8dad03e2, user_id=e237e18bd09e47d897635c34512844cd, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:04.178715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87aaf3-7aee-4421-a46b-8c0d28848041 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:04.205272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 588/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:11:03 2022] PUT /identity/v3/projects/705e0f35646f490394d9448a8dad03e2/users/e237e18bd09e47d897635c34512844cd/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:11:06.482760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651015e0-50e1-4681-adf8-247e2ab144fa None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:06.483620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651015e0-50e1-4681-adf8-247e2ab144fa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:06.483800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651015e0-50e1-4681-adf8-247e2ab144fa None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:06.486390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 605/1193] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:11:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:11:06.494882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:06.495437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:06.495838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:06.528204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:11:06.530325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:11:06.618810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:06.639065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 589/1194] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:11:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:11:06.663913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651015e0-50e1-4681-adf8-247e2ab144fa None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:06.814153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651015e0-50e1-4681-adf8-247e2ab144fa None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:06.855168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:06.856001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:06.856287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:06.856561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:06.869026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:11:06.877070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:11:06.878390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:11:06.879509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:11:06.882096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 606/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:11:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:11:06.885993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:07.026356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ae6e-59c9-4e32-928d-42915a29ff2a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:07.075933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:07.076696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:07.077044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:07.077383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:07.078171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:11:07.078606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:07.080137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:07.093140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 590/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:11:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:07.097599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:07.120726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:07.248450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c244-da8a-44b5-a10b-a27500193867 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:07.289093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:07.290036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:07.290321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:07.290599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] PATH_INFO: `/v3/users/e237e18bd09e47d897635c34512844cd` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:07.291898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] RBAC: Authorizing `identity:get_user(user_id=e237e18bd09e47d897635c34512844cd)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:07.293457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:07.295250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 607/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 25 16:11:07 2022] GET /identity/v3/users/e237e18bd09e47d897635c34512844cd => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:07.302338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:07.371910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:07.425670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfdb0b-9fd9-44e0-9866-0b6c0407358b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:07.471948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:07.472636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:07.472913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:07.473188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:07.560080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:07.561825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:07.630606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:07.680738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 591/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:11:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:11:07.687056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:07.787878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:07.824434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247adea5-ffd2-4750-93f1-0a9dcfb1a7ee None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:07.869801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:07.870718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:07.871006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:07.871280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] PATH_INFO: `/v3/projects/705e0f35646f490394d9448a8dad03e2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:07.872810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] RBAC: Authorizing `identity:get_project(project_id=705e0f35646f490394d9448a8dad03e2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:07.874609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe35ac1-6e11-4b5e-8f80-d242c372ba42 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:07.876770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 608/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 25 16:11:07 2022] GET /identity/v3/projects/705e0f35646f490394d9448a8dad03e2 => generated 361 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:11:07.887316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:11:08.062133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9adc3-8b8e-426d-84e5-928b42483e8d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:11:08.132373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:11:08.133617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:11:08.134108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:11:08.134591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:11:08.136258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=705e0f35646f490394d9448a8dad03e2, user.id=e237e18bd09e47d897635c34512844cd) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:11:08.136815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:11:08.139838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d53b5d-af2c-42a6-b457-5169f765ee48 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:11:08.153163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 592/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:11:07 2022] GET /identity/v3/role_assignments?user.id=e237e18bd09e47d897635c34512844cd&scope.project.id=705e0f35646f490394d9448a8dad03e2&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:11:17.864694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=25 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:11:18.146289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:12.804450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53746c6-fe99-404e-b0d9-3ecda1d41501 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:12.804450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53746c6-fe99-404e-b0d9-3ecda1d41501 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:12.804450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53746c6-fe99-404e-b0d9-3ecda1d41501 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:12.807083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 609/1201] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:12:12.814442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:12.815337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:12.815337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:12.847968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:12.850683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:13.012189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 593/1202] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:12:13.042598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53746c6-fe99-404e-b0d9-3ecda1d41501 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:13.281434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53746c6-fe99-404e-b0d9-3ecda1d41501 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:13.356932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:13.358320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:13.358765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:13.359244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:13.381907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:13.384436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:13.420731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:13.423052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:13.425077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:13.425077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:13.425077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:13.425077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:13.428400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 610/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:13.433152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:13.664020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed449a95-6d69-41e9-928b-745fc102c364 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:13.723849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:13.734495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:13.735566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:13.736024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:13.736557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:13.737573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:13.738099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:13.740625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9ab22-8880-430d-bb57-85fd2604e8da None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:13.764372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 594/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:13.775761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:13.890808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:13.991832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccb9ff-5cd8-4930-9f2e-1c63524a18da None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:14.064893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:14.066221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:14.066713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:14.067193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:14.068430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:14.071446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684559c3-42b3-485d-9775-bebfc5884ad8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:14.119307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 611/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 25 16:12:13 2022] POST /identity/v3/users => generated 251 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:12:16.955621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd256c-7fa7-4f5e-9735-969e789dcd9c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:16.956236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd256c-7fa7-4f5e-9735-969e789dcd9c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:16.956570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd256c-7fa7-4f5e-9735-969e789dcd9c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:16.958499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 595/1206] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:12:16.966372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7e048-d774-45df-a338-1040df71b82c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:16.967012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7e048-d774-45df-a338-1040df71b82c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:16.967336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7e048-d774-45df-a338-1040df71b82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:16.994073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aca7e048-d774-45df-a338-1040df71b82c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:16.996306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aca7e048-d774-45df-a338-1040df71b82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:17.111381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 612/1207] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:17.148663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd256c-7fa7-4f5e-9735-969e789dcd9c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:17.329751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd256c-7fa7-4f5e-9735-969e789dcd9c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:17.402138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:17.403543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:17.404050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:17.404515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:17.426193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:17.439442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:17.441623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:17.444319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:17.447782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 596/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:17.458719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7e048-d774-45df-a338-1040df71b82c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:17.536363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:17.653761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7e048-d774-45df-a338-1040df71b82c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:17.714593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:17.715624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:17.716105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:17.716672 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:17.717744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:17.718225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:17.719951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:17.735777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 613/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:12:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:17.744129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:17.847707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:17.930580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881723a-c541-4ef2-a6d8-6b74d6b81c0d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:17.997084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:17.998495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:17.998955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:18.000252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:18.020219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:18.023105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:18.058565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:18.059680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:18.061109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:18.061109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:18.061109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:18.061109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:18.061109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 597/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:18.067117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:18.238494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:18.247402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435611ad-77a1-4282-9a14-001f672cf731 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:18.306488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:18.307927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:18.308533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:18.308985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:18.310465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:18.311269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:18.313861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:18.337804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 614/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:18.345847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:18.507027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c778e0-84cc-423b-9508-02d1e836f0ac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:18.509206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:18.565138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a749aae-0760-452e-b831-e03104771724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:18.566599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749aae-0760-452e-b831-e03104771724 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:18.567078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749aae-0760-452e-b831-e03104771724 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:18.567557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749aae-0760-452e-b831-e03104771724 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:18.581770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a749aae-0760-452e-b831-e03104771724 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:18.584264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a749aae-0760-452e-b831-e03104771724 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4a749aae-0760-452e-b831-e03104771724 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:18.607840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:18.609862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:18.617118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 598/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:18.618228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:18.804081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896625f3-032d-4d3a-af2e-2a9f14c82ee4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:18.879724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:18.880605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:18.880867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:18.881184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:18.883176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:18.883553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:18.885457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:18.916025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 615/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:18.924069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a749aae-0760-452e-b831-e03104771724 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:19.062836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a749aae-0760-452e-b831-e03104771724 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:19.122317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:19.124154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:19.125072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:19.125923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:19.195740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:19.275934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:19.278575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:19.461106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 599/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:12:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:12:19.468848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:19.507223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:19.670363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ee209-e4f7-4cb3-bbc9-7ca88497e250 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:19.737452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:19.738955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:19.739537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:19.739983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:19.757880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:19.761032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:19.797357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:19.799155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:19.801027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:19.801027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:19.801027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:19.801027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:19.801027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 616/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:19.810641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:19.928326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:19.996860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5073b3-4523-4eb8-910b-19420ea2dd82 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:20.073040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:20.074214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:20.074612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:20.075134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:20.076131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:20.076573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:20.078447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:20.101709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 600/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:20.108591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:20.269864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:20.333051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652b88d-cecb-497d-80c0-7797038a5413 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:20.412568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:20.414290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:20.414906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:20.415548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:20.430148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:20.432953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:20.466919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:20.468097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:20.474280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 617/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:20.481880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:20.692460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c904-2e30-4f71-9191-10bce51fd1af None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:20.745048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:20.770169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:20.771659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:20.772071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:20.772448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:20.773758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:20.774184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:20.776277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e112ed4-d7a1-4696-8027-4cc2e972c84f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:20.805700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 601/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:20.808420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:20.984244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c36d4-b7d0-43f3-a6a1-fdf754afba68 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:21.033496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:21.043840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:21.044806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:21.045324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:21.045656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:21.046626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:21.046941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:21.048738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5bf5-ab76-4921-9056-b766c6d64967 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:21.057832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 618/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:12:20 2022] GET /identity/v3/role_assignments?user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:23.473720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd776f6-29bf-4b80-9257-eee9e11dfd43 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:23.474919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd776f6-29bf-4b80-9257-eee9e11dfd43 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:23.474919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd776f6-29bf-4b80-9257-eee9e11dfd43 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:23.475929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 602/1220] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:12:23.482393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:23.483280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:23.483467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:23.512354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:23.514271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:23.621034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 619/1221] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:12:23.646571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd776f6-29bf-4b80-9257-eee9e11dfd43 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:23.821981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd776f6-29bf-4b80-9257-eee9e11dfd43 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:23.882212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:23.883400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:23.883883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:23.884339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:23.908005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:23.924624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:23.926745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:23.928726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 603/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:12:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:23.934047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:24.066160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba773a34-db2b-426d-8a0b-a03a19728c07 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:24.097181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:24.112180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:24.112891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:24.113177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:24.113597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:24.114398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:24.114753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:24.116251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:24.127839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 620/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:12:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:24.137652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:24.223060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:24.323495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbbc93-2fd8-45e5-af77-7a31a02190d4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:24.387270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:24.388506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:24.389013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:24.389548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:24.413112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:24.415210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:24.437659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:24.438860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:24.440852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:24.440852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:24.440852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:24.440852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:24.440852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 604/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:24.448330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:24.636650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5acf-8c45-48ab-9411-027075e147d3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:24.704055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:24.705046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:24.705430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:24.705792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:24.706767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:24.707186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:24.708821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:24.729105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 621/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:24.730934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:24.927361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:24.938193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e67e96-91e1-4d78-9d50-1a5485876273 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:24.990447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:24.991359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:24.991679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:24.991956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:25.002011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:25.003636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:25.021544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:25.023730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:25.025663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 605/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:25.030852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:25.101882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:25.219914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105463-8edf-4470-9321-940778ec9a48 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:25.277110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:25.277826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:25.278272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:25.278605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:25.279888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:25.280205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:25.282086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:25.305833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 622/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:25.316168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:25.490384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a1a0-4cea-42d0-85c0-bd2dd469e129 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:25.555842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:25.556828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:25.557472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:25.558085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:25.581680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:25.584787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:25.608999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:25.616166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:25.617117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:25.617117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:25.617117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:25.617117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:25.617117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 606/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:25.617469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:25.820524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c066b3-8057-44f4-8551-895c3051921c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:25.824343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:25.894838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:25.897166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:25.897936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:25.898688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:25.900610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:25.900964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:25.904519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:25.928703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 623/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:25.936293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:25.955501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:26.135900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea3d78-f2ec-4259-8be8-05220c41644f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:26.212860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:26.213913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:26.214369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:26.214591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:26.224377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:26.226842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:26.260022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:26.261755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:26.263070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 607/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:26.269947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:26.425932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821b58-242a-48a5-91b1-58e9fd5821e5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:26.475358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:26.476103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:26.476384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:26.476655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:26.477818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:26.478136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:26.479958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835819-92f3-4c30-9665-789a320eaad0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:26.495933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 624/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:26.502676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:26.623239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:26.683526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5652-0881-49e5-bd18-1b9383e68fd1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:26.750409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:26.752846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:26.752846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:26.753189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/ecf1c53cb8784f5c9a2d8d2d71751fa2/roles/e6c14c02cd4944e48c25d4780a573718` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:26.778177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=ecf1c53cb8784f5c9a2d8d2d71751fa2, role_id=e6c14c02cd4944e48c25d4780a573718)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:26.780317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d053-c30b-4f1c-bebd-8fb5b1182769 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:26.800683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 608/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:12:26 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/ecf1c53cb8784f5c9a2d8d2d71751fa2/roles/e6c14c02cd4944e48c25d4780a573718 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:12:29.110937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac190a1-3207-41ac-94ac-48befd40625d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:29.111780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac190a1-3207-41ac-94ac-48befd40625d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:29.111780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac190a1-3207-41ac-94ac-48befd40625d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:29.113948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 625/1233] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:12:29.121765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:29.125316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:29.125911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:29.157281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:29.159544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:29.286914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 609/1234] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:29.317065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac190a1-3207-41ac-94ac-48befd40625d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:29.360930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:29.509673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac190a1-3207-41ac-94ac-48befd40625d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:29.554470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:29.555311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:29.555576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:29.555873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:29.569026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:29.577792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:29.579707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:29.581379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 626/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 25 16:12:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:29.586914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:29.664940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:29.765881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7429b6-c8fc-45f1-8d2c-1bd4c8f18363 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:29.829427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:29.830290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:29.830713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:29.831155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:29.832400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:29.832844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:29.835178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:29.850291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 610/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 25 16:12:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:29.860628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:30.029929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:30.066259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd77508-ca3b-4da7-b258-ae9145ee8cbe None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:30.134947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:30.136385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:30.136878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:30.137382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:30.163435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:30.167933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:30.211504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:30.212668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:30.213903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:30.213903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:30.213903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:30.213903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:30.214325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 627/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:30.222849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:30.398889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4635b8-3898-4d86-875f-58bffcf2e686 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:30.411490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:30.458952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:30.460095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:30.460654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:30.461107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:30.462584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:30.463137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:30.465614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:30.487991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 611/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:30.494689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:30.681438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0f36-9f27-4f23-9737-3beed02a5535 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:30.727113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:30.727459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:30.727748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:30.728024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:30.731594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:30.741568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:30.743127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:30.765864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:30.767400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:30.768613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:30.773407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 628/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:30.773962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:30.954821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fffb2-d4be-45b8-b283-a13b1de39f00 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:31.010568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:31.011541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:31.011941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:31.012459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:31.013578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:31.014025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:31.015699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:31.042624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 612/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:31.050271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:31.229089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b382b1-df4e-4a2a-bf12-d25257c98e8b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:31.280795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:31.282058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:31.282549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:31.283045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:31.388853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:31.400764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:31.403289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:31.590618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 629/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:12:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:12:31.598441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:31.643984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:31.773443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b738b42-d11d-439c-8269-602c785c089e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:31.841323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:31.842877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:31.843510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:31.844108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:31.863425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:31.866309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:31.900599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:31.902688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:31.904881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:31.904881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:31.904881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:31.904881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:31.904881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 613/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:31.912135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:32.105308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:32.116133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605529ba-176a-40bd-a6cd-b0a8b03c50af None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:32.195447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:32.197206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:32.197644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:32.198363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:32.200117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:32.200832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:32.204213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:32.230519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 630/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:32.238403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:32.357198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:32.390936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da55c6-901d-43f7-af50-22f0ed2c4df8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:32.446163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:32.447536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:32.448021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:32.448493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:32.459040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:32.462365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:32.486248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:32.487573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:32.488601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 614/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:32.497129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:32.627355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee726ba-d61f-46ce-ae0e-80e1f9b65e89 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:32.676982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:32.677852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:32.678141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:32.678311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:32.679116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:32.679469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:32.681013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8f812-ce56-4775-bd29-3c0b86e01902 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:32.696981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 631/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:32.704759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:32.881097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffdda7-6ea6-42dc-a6c3-2fd8897ccc42 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:32.935029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:32.935844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:32.937267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:32.937267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:32.937643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] RBAC: Adding query filter params (role.id=e6c14c02cd4944e48c25d4780a573718, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:32.938051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:32.939771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88a4dd-a809-4482-99e3-aa037e719b69 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:32.949408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 615/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:12:32 2022] GET /identity/v3/role_assignments?user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=e6c14c02cd4944e48c25d4780a573718 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:35.226563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a10f-4233-4f3c-bedf-55fb0d497cec None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:35.227732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a10f-4233-4f3c-bedf-55fb0d497cec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:35.227732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a10f-4233-4f3c-bedf-55fb0d497cec None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:35.228598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 632/1247] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:12:35.238583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:35.239450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:35.239618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:35.250898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:35.274899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:35.277006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:35.401386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 616/1248] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:35.425602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978a10f-4233-4f3c-bedf-55fb0d497cec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:35.457230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:35.595689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978a10f-4233-4f3c-bedf-55fb0d497cec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:35.652036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:35.653202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:35.653684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:35.654102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:35.672099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:35.682847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:35.685045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:35.686157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 633/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:35.692509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:35.834698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b34-564c-4b4e-8d0e-d31fa0bb3db4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:35.880847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:35.890460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:35.891654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:35.892149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:35.892607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:35.894082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:35.894624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:35.897266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:35.914566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 617/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:35.921975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:36.046280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:36.094485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bab45b-d51e-4004-9afb-fa76a69ab391 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:36.154690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:36.155736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:36.156159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:36.156534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:36.181235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:36.184341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:36.215973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:36.218027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:36.219583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:36.219583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:36.219583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:36.219583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:36.219583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 634/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:36.227663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:36.409207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe22ba-f8fd-4171-9b34-3f7436ca8a43 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:36.472462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:36.473719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:36.474288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:36.474800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:36.476277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:36.476861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:36.479538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:36.499028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 618/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:36.506838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:36.674332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb075d-e4df-466e-8f0f-dc03ff00bb3b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:36.721468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:36.737001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:36.738740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:36.739184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:36.739655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:36.755124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:36.757435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:36.787087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:36.789372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:36.791577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 635/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:36.796256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:36.889485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:36.957861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee47d9-ec81-477f-9a52-7f7293ccfde8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:37.009288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:37.010035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:37.010322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:37.010594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:37.011607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:37.011951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:37.013565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:37.036999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 619/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:37.043249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:37.216642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da32c49-fe4b-4e04-a901-292291368606 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:37.267362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:37.268198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:37.268522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:37.268901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:37.351885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:37.369734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:37.371469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:37.541577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 636/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:12:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 16:12:37.548034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:37.739160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e77f9f-6bf9-4673-a37e-c9a1edb6e183 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:37.757990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:37.797575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:37.798699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:37.798816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:37.799193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:37.817370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:37.819378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:37.845247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:37.847191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:37.849120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:37.849120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:37.849120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:37.849120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:37.849120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 620/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:37.854915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:38.021471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:38.047036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27952-a331-46e7-a073-ffa4eea355e6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:38.109920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:38.110704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:38.111048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:38.111382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:38.112242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:38.112699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:38.114691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:38.135632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 637/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:38.144500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:38.382414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9f61-73f5-430a-821b-7d392de13449 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:38.435358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:38.436209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:38.436493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:38.436879 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:38.445126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:38.446649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:38.463454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:38.464807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:38.465758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 621/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:38.471835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:38.621906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8901-cc49-4953-beac-744dcf7bd978 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:38.674134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:38.675519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:38.675987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:38.676429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:38.677856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:38.678388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:38.681445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb4718-81fb-438b-b6c9-4ca06664b5f3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:38.702688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 638/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:38.712860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:38.746288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:38.851823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5b2f0-0423-48f4-bee2-c6e1a140edc4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:38.895143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:38.895867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:38.896150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:38.896428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:38.897292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:38.897703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:38.899320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e397-cda0-4047-9ebf-39eaec7a063c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:38.909387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 622/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:12:38 2022] GET /identity/v3/role_assignments?user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:41.347509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5c535-0e3d-4712-b367-683dbb8d6954 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:41.348425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5c535-0e3d-4712-b367-683dbb8d6954 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:41.348425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5c535-0e3d-4712-b367-683dbb8d6954 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:41.350177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 639/1261] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:12:41.361302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:41.362105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:41.362105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:41.386937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:41.388299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:41.494684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 623/1262] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:41.535871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5c535-0e3d-4712-b367-683dbb8d6954 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:41.557976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:41.731269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5c535-0e3d-4712-b367-683dbb8d6954 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:41.777967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:41.779007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:41.779475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:41.779842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:41.793754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:41.802816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:41.804409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:41.806126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 640/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:41.813482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:41.840031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:41.968707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1b8c7-7169-4648-bbee-a869ef641035 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:42.020251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:42.020855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:42.021139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:42.021494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:42.022577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:42.022896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:42.024391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:42.035917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 624/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:42.042391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:42.171070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:42.244379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57319a0-9fbc-429a-9146-936a2345260e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:42.305310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:42.306656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:42.307136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:42.307598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:42.329137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:42.332034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:42.366317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:42.367953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:42.369070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:42.369070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:42.369070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:42.369070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:42.369070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 641/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:42.378759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:42.486729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:42.512493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37597b-65d6-4890-898e-f3f8876b115f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:42.552394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:42.553147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:42.553467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:42.553745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:42.554526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:42.554812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:42.556334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:42.570876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 625/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:42.577527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:42.700075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f16d4-3464-49b2-9058-a6f7ebf86349 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:42.732117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:42.747802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:42.748648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:42.748932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:42.749207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:42.758800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:42.760407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:42.777209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:42.779428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:42.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 642/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:42.787175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:42.912970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101c124-cb3c-4234-a9e9-775f96fed653 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:42.952607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:42.953592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:42.953924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:42.954248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:42.955167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:42.955520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:42.957065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:42.968270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:42.979371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 626/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:42.985861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:43.110410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb0b0d-cf43-4313-bf2d-bfef73566dfb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:43.154979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:43.155869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:43.156205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:43.156531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:43.175563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:43.178367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:43.205412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:43.206617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:43.208465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:43.208465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:43.208465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:43.208465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:43.208465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 643/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:43.213277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:43.346000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824263e7-843a-47f3-b187-19c5f2c32feb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:43.400084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:43.401256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:43.401816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:43.402294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:43.403756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:43.404275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:43.407121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:43.433950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 627/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:43.443497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:43.507872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:43.625926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429e49-660c-4fa3-81a4-cfaaad87a868 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:43.682452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:43.683834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:43.684385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:43.684879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:43.698743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:43.701721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:43.730631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:43.732498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:43.734651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 644/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:43.742901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:43.830802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:43.881716 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c9be-3d44-4e32-9f16-ca2b109b0a91 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:43.921396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:43.922250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:43.922577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:43.922901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:43.923966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:43.924321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:43.925998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf317c-5224-4f3f-81f0-a91c9c6eddd5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:43.941027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 628/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:43.948081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:44.100242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:44.116151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc13ad-40e3-4da2-b64e-48a82c3758d2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:44.206852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:44.208726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:44.209508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:44.210181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] PATH_INFO: `/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/ecf1c53cb8784f5c9a2d8d2d71751fa2/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:44.215452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0729a317dec4f6a9639c0492b388eba, user_id=ecf1c53cb8784f5c9a2d8d2d71751fa2, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:44.218266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7020f0-aa14-4884-b5b6-d86dd5046d67 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:44.242136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 645/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 25 16:12:43 2022] PUT /identity/v3/projects/c0729a317dec4f6a9639c0492b388eba/users/ecf1c53cb8784f5c9a2d8d2d71751fa2/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:12:46.407677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af94a0-7e3d-4dc8-a8bc-38d00b4c28ea None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:46.408276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af94a0-7e3d-4dc8-a8bc-38d00b4c28ea None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:46.408633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af94a0-7e3d-4dc8-a8bc-38d00b4c28ea None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:46.411377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 629/1274] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:12:46.418895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:46.419789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:46.419789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:46.443182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:46.443774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:46.532554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 646/1275] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:12:46.556663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af94a0-7e3d-4dc8-a8bc-38d00b4c28ea None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:46.744935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af94a0-7e3d-4dc8-a8bc-38d00b4c28ea None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:46.775917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:46.808035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:46.809479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:46.809979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:46.810471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:46.828303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 16:12:46.836756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:46.841689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:46.843363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 630/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 25 16:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 16:12:46.844995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:46.994171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:47.022108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bb3e7-beb3-4bc8-8661-254213144244 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:47.065939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:47.066633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:47.066923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:47.067205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:47.067982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:47.068292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:47.069789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:47.080566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 647/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 25 16:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:47.086888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:47.244986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007481d-a8a9-484b-b28a-ae75b0024e90 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:47.289329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:47.290109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:47.290393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:47.290779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:47.303930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:47.305469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:47.325040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:47.326900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:47.328881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:47.328881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:47.328881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:47.328881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:47.328881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 631/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:47.338224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:47.474101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421210-fabc-48c0-ad4a-33ff3dfa36f9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:47.521689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:47.522956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:47.523460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:47.523881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:47.525329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:47.525923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:47.528660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:47.552456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 648/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:47.558686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:47.598179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:47.685092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dcf5-4f04-4464-bfc7-42fb4ce9f631 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:47.730152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:47.731008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:47.731294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:47.731567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:47.745231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:47.748049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 16:12:47.777211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:47.779055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:47.780730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 632/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:47.789980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:47.797216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:47.927498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a64a-4c23-4af4-8c5a-8b4ac093ba44 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:47.984926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:47.985836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:47.986203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:47.986563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:47.987531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:47.987928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:47.989627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:48.018342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 649/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 25 16:12:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:48.028183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:48.206917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:48.220930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbfa9-ada4-4a81-820d-ba26130e17aa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:48.269156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:48.270019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:48.270340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:48.270776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:48.375481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:48.377032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:48.493673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:48.507478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 633/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 25 16:12:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 16:12:48.514489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:48.636012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:48.711698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175d74-3a0d-4524-9f08-e58e5dbbfee4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:48.779662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:48.780921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:48.781514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:48.782026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:48.801994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:48.804566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 16:12:48.838794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:48.839946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:48.841094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 16:12:48.841094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 16:12:48.841094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 16:12:48.841094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:48.841094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 650/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:48.851594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:49.004256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0425f-bec2-4a09-9d79-8aff8b5d1b08 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:49.055356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:49.056021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:49.056346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:49.056665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:49.057541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:49.057895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:49.059404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:49.073451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 634/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 25 16:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:49.083420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:49.266608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e716f0-d9a0-4b49-aa51-f674214fcebb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:49.297087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:49.338448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:49.339767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:49.340259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:49.340736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:49.354735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:49.357643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 16:12:49.385881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:49.387661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 25 16:12:49.389629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 651/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 25 16:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:49.398641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:49.494103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:49.569702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44a8d-f0ca-4b98-9c9d-51471cb7620d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:49.619869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:49.620622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:49.621080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:49.621602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:49.622578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:49.622983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:49.624575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0916-adf6-45aa-bd44-848020a9eaf6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:49.642109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 635/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 25 16:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:49.652519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:49.851351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817a21e-6c21-4d3e-9237-08b60bcb5881 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:49.897718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:49.898385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:49.898721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:49.899130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:49.900126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] RBAC: Adding query filter params (role.id=8b8b7516fb6f40bd878cd90fb40874ea, scope.project.id=c0729a317dec4f6a9639c0492b388eba, user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:49.900491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:49.902156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8cb9d-6f36-4d60-8df2-dc3e09073097 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:49.910258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 652/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 25 16:12:49 2022] GET /identity/v3/role_assignments?user.id=ecf1c53cb8784f5c9a2d8d2d71751fa2&scope.project.id=c0729a317dec4f6a9639c0492b388eba&role.id=8b8b7516fb6f40bd878cd90fb40874ea => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:52.221504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44891-a4af-4e48-b3e5-69ee5a5f128a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:52.222440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44891-a4af-4e48-b3e5-69ee5a5f128a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:52.222440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44891-a4af-4e48-b3e5-69ee5a5f128a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:52.224366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 636/1288] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:12:52.232815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712f68-aced-4322-b96d-951c3becbcde None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:52.233580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712f68-aced-4322-b96d-951c3becbcde None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:52.234008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712f68-aced-4322-b96d-951c3becbcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:52.274597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-49712f68-aced-4322-b96d-951c3becbcde None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:52.277856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-49712f68-aced-4322-b96d-951c3becbcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:52.341655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:52.424069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 653/1289] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:52.462885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc44891-a4af-4e48-b3e5-69ee5a5f128a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:52.635743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc44891-a4af-4e48-b3e5-69ee5a5f128a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:52.650111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:52.686170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:52.687098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:52.687380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:52.687661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:52.688343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:52.689917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:52.704365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:52.706549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:52.708554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 637/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:12:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:52.714392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712f68-aced-4322-b96d-951c3becbcde None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:52.838747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712f68-aced-4322-b96d-951c3becbcde None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:52.880651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:52.881804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:52.882076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:52.882354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:52.883095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:52.883417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:52.884859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839ceff-506a-4661-bb15-af30ece4f54c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:52.890991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 654/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 25 16:12:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:12:52.898266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:53.046162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9f4ba-7794-4b7f-87dc-fad403ff0e10 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:53.107745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:53.108706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:53.109200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:53.109723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:53.111129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:53.111691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:53.114458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9dd04-6315-4a83-9581-89fdb025f995 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:53.123708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 638/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 25 16:12:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:55.466078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665dd37-35df-45b4-bb12-e1482477bdb7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:55.466955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665dd37-35df-45b4-bb12-e1482477bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:55.466955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665dd37-35df-45b4-bb12-e1482477bdb7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:55.468844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 655/1293] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:12:55.475980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2d51-313e-42c6-a27a-8a1dfc16a432 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:55.476868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2d51-313e-42c6-a27a-8a1dfc16a432 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:55.476868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2d51-313e-42c6-a27a-8a1dfc16a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:55.505016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b32a2d51-313e-42c6-a27a-8a1dfc16a432 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:55.507415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b32a2d51-313e-42c6-a27a-8a1dfc16a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:55.562894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:55.631134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 639/1294] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 16:12:55.654343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665dd37-35df-45b4-bb12-e1482477bdb7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:55.694628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:55.789309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665dd37-35df-45b4-bb12-e1482477bdb7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:55.833588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:55.834359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:55.834681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:55.835043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:55.835805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:55.837291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f7023-5688-43a1-9281-4b8f9e91d403 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:55.856929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 656/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 25 16:12:55 2022] POST /identity/v3/services => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:12:58.120141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7049a-b97b-48ff-9d6a-83031fa5ebac None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:58.120792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7049a-b97b-48ff-9d6a-83031fa5ebac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:58.120995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7049a-b97b-48ff-9d6a-83031fa5ebac None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:58.123567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 640/1296] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:12:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:12:58.133857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:58.134780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:58.135136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:58.164111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:12:58.165979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:12:58.295973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 657/1297] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:12:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:12:58.319785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7049a-b97b-48ff-9d6a-83031fa5ebac None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:58.483804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7049a-b97b-48ff-9d6a-83031fa5ebac None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:58.564146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:58.565106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:58.565595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:58.566049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:58.566899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:58.568872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:12:58.587769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:12:58.589862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:12:58.591999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 641/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:12:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:12:58.600313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:58.635785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:58.756290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedee8c5-4413-47c6-a541-81df059770d8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:58.819776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:58.820599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:58.820873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:58.821158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:58.821916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:58.822228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:58.823678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:58.830120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 658/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 25 16:12:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:58.837043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:58.903443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:59.045624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391d4f-f540-4080-a972-4b6bebc9c812 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:59.112988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:59.113866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:59.114305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:59.114714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:59.115976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] RBAC: Adding query filter params (interface=public, service_id=c3c11fa0e1134c45b1dd81175d953271, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:12:59.116295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:59.119011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8ec7-71cd-4826-9267-3aaae0a527ea None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:59.128486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 642/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 25 16:12:58 2022] GET /identity/v3/endpoints?service_id=c3c11fa0e1134c45b1dd81175d953271&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:12:59.137621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:12:59.251291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:12:59.289358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969018a-b588-4d01-ba19-83eef6a9196a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:12:59.346629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:12:59.347212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:12:59.347493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:12:59.347786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:12:59.348642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:12:59.350562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41a2df-4376-4b2a-8a8c-633e47f9fb7c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:12:59.359035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 659/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 25 16:12:59 2022] GET /identity/v3/services => generated 2191 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:13:01.901215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faea6a-d908-4d50-9200-560f6632667e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:13:01.902203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faea6a-d908-4d50-9200-560f6632667e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:13:01.902203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faea6a-d908-4d50-9200-560f6632667e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:13:01.904582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 643/1302] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:13:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:13:01.914889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:13:01.915711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:13:01.915866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:13:01.945448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:13:01.946953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:13:02.043670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 660/1303] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:13:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 16:13:02.067708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faea6a-d908-4d50-9200-560f6632667e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:13:02.250412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:13:02.264341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faea6a-d908-4d50-9200-560f6632667e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:13:02.328632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:13:02.329912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:13:02.330322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:13:02.330741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:13:02.331779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:13:02.334097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 16:13:02.355557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 16:13:02.357590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 25 16:13:02.358791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 644/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 25 16:13:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 16:13:02.366953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:13:02.478476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:13:02.514641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666334-693d-4bb9-b7d2-f59eaada54a0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:13:02.557715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:13:02.558420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:13:02.558706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:13:02.558982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:13:02.559730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:13:02.560038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:13:02.561603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998ac1a-8d92-474c-935a-3b9f20d935e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:13:02.568292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 661/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 25 16:13:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:13:02.578378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:13:02.747411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9519-241c-4022-ad0e-b09b7ce2317c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:13:02.813042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:13:02.813696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:13:02.814064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:13:02.814350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:13:02.815260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:13:02.816751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353f2a7-6fff-4753-8fa6-fe7c5ad33907 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:13:02.848528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 645/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:13:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:13:12.564618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=30 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:13:12.839261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:20.240677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d08f-e49a-433b-b26e-68173e248116 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:20.242074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d08f-e49a-433b-b26e-68173e248116 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:20.242736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d08f-e49a-433b-b26e-68173e248116 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:20.245304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 662/1307] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:20.251306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694371-7dfc-45df-a912-21126da7959b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:20.251974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694371-7dfc-45df-a912-21126da7959b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:20.251974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3694371-7dfc-45df-a912-21126da7959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:20.378806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f3694371-7dfc-45df-a912-21126da7959b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:20.380637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f3694371-7dfc-45df-a912-21126da7959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:20.536557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 646/1308] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:20.610681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:20.611302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:20.612013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:20.616675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 663/1309] 10.222.0.66 () {54 vars in 965 bytes} [Mon Jul 25 16:14:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:20.629995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd6026-c2e6-4a07-b053-a894dbd3838f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:20.630370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd6026-c2e6-4a07-b053-a894dbd3838f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:20.630667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd6026-c2e6-4a07-b053-a894dbd3838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:20.733614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-60dd6026-c2e6-4a07-b053-a894dbd3838f None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:20.735176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-60dd6026-c2e6-4a07-b053-a894dbd3838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:20.764030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:20.908155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 647/1310] 10.222.0.66 () {58 vars in 1062 bytes} [Mon Jul 25 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:14:20.930864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:20.932773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:21.081976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0789c-9686-4025-9040-ad5c0d44fe07 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:21.213416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:21.214552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:21.215067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:21.215553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:21.218838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:21.309915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:21.393090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:21.394716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcdace-8e4c-4cdd-96b9-255d03a1ae7d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:21.421941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 664/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:14:25.463786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b39a54-1698-4259-910e-759d88d8116c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:25.464552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b39a54-1698-4259-910e-759d88d8116c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:25.464740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b39a54-1698-4259-910e-759d88d8116c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:25.467715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 648/1312] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:25.476622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644101c7-9b0c-4d1a-b2c9-c74ad88c0f9c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:25.477270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644101c7-9b0c-4d1a-b2c9-c74ad88c0f9c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:25.477654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644101c7-9b0c-4d1a-b2c9-c74ad88c0f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:25.584335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-644101c7-9b0c-4d1a-b2c9-c74ad88c0f9c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:25.585641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-644101c7-9b0c-4d1a-b2c9-c74ad88c0f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:25.707339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 665/1313] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:25.766593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:25.766907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:25.767231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:25.769155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 649/1314] 10.222.0.66 () {54 vars in 965 bytes} [Mon Jul 25 16:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:25.776985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532fd8b-e58b-4841-9813-c2dbe92d555e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:25.777315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532fd8b-e58b-4841-9813-c2dbe92d555e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:25.777746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532fd8b-e58b-4841-9813-c2dbe92d555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:25.791153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:25.889792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2532fd8b-e58b-4841-9813-c2dbe92d555e None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:25.891192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2532fd8b-e58b-4841-9813-c2dbe92d555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:25.987432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 666/1315] 10.222.0.66 () {58 vars in 1062 bytes} [Mon Jul 25 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:25.997699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:25.999485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:26.138358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2458-8a7b-45c6-991e-2e75532b0043 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:26.232543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:26.236169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:26.236997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:26.237790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:26.238134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:26.240743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:26.395863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:26.398619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760df4e6-875a-4d51-8807-fd3ffeb3eb81 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:26.438942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 650/1316] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:29.232219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:29.232890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:29.232890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:29.234286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 667/1317] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:29.241887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f0211-ad10-4891-a0b8-acbe39a8c0bd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:29.242474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f0211-ad10-4891-a0b8-acbe39a8c0bd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:29.242915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f0211-ad10-4891-a0b8-acbe39a8c0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:29.350251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4e9f0211-ad10-4891-a0b8-acbe39a8c0bd None None] MFA Rules not processed for user `dded27d91b9449209b35a9c238089d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:29.352981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4e9f0211-ad10-4891-a0b8-acbe39a8c0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:29.432474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:29.519029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 651/1318] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:29.588492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:29.590168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:29.774119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658eafd-da81-4999-9f94-5772f232c059 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:29.778287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:29.909931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:29.910706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:29.910987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:29.911260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:29.913270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:30.084893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:30.086890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74e27a-6e6b-4758-844f-4a66fe68f464 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:30.114580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 668/1319] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:33.428877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:33.429576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:33.429576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:33.431063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 652/1320] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:33.438042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937abe9-41e6-4d6f-9c99-d649876fc1f5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:33.438702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937abe9-41e6-4d6f-9c99-d649876fc1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:33.438702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937abe9-41e6-4d6f-9c99-d649876fc1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:33.499960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:33.547303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5937abe9-41e6-4d6f-9c99-d649876fc1f5 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:33.549434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5937abe9-41e6-4d6f-9c99-d649876fc1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:33.722124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 669/1321] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:33.767038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:33.768686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:33.956960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d0fe5-628d-4676-b877-71a6bcd55355 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:34.106919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:34.108038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:34.108536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:34.109028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:34.112463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:34.128297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:34.331164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:34.334143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325b5dc-95ce-4137-913c-9d905d6f143d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:34.372725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 653/1322] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:38.146367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15328145-716b-467f-b078-ad1534b9c200 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:38.147173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15328145-716b-467f-b078-ad1534b9c200 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:38.147173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15328145-716b-467f-b078-ad1534b9c200 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:38.148742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 670/1323] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:38.155104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104efd25-8f45-4b64-8b11-80595c3c0494 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:38.155952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104efd25-8f45-4b64-8b11-80595c3c0494 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:38.155952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104efd25-8f45-4b64-8b11-80595c3c0494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:38.266877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-104efd25-8f45-4b64-8b11-80595c3c0494 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:38.268961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-104efd25-8f45-4b64-8b11-80595c3c0494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:38.372007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:38.448278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 654/1324] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:14:38.489305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15328145-716b-467f-b078-ad1534b9c200 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:38.491766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-15328145-716b-467f-b078-ad1534b9c200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:38.624165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15328145-716b-467f-b078-ad1534b9c200 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:38.640852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:38.724146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:38.724909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:38.725191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:38.725504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:38.727308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:38.907185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:38.910237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e967854-4a67-4883-848b-d9a37282becd service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:38.942439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 671/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:41.953479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:41.954262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:41.954262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:41.955779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 655/1326] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:41.965631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f34449-f88d-4acd-b629-9e71f961e209 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:41.966436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f34449-f88d-4acd-b629-9e71f961e209 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:41.966591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f34449-f88d-4acd-b629-9e71f961e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:42.060995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:42.097667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d4f34449-f88d-4acd-b629-9e71f961e209 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:42.099839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d4f34449-f88d-4acd-b629-9e71f961e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:42.285078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 672/1327] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:42.338926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:42.340709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:42.487067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea96f0-b30b-4c64-b298-7239ddd60f69 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:42.612509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:42.613287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:42.613651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:42.613966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:42.615940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:42.649751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:42.839631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:42.842452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137ab5-44e4-4716-a611-96e6f5b33a7a service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:42.873475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 656/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:45.316336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:45.317183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:45.317371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:45.318870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 673/1329] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:45.327801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8ad8-bc6c-468c-9b35-ac07016d3cc1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:45.328666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8ad8-bc6c-468c-9b35-ac07016d3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:45.328666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8ad8-bc6c-468c-9b35-ac07016d3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:45.416804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0eca8ad8-bc6c-468c-9b35-ac07016d3cc1 None None] MFA Rules not processed for user `dded27d91b9449209b35a9c238089d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:45.418226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0eca8ad8-bc6c-468c-9b35-ac07016d3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:45.528207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:45.576557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 657/1330] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:14:45.626136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:45.628523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:45.816695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22b80-9b00-4d4b-9476-517eb2cfebbe None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:45.859817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:45.978702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:45.980034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:45.980644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:45.981157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:45.984717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:46.246489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:46.250025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47d4ee-b0d6-41c7-ab4f-3f28706d1cbf service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:46.288303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 674/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:14:49.188630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:49.189500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:49.189500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:49.191772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 658/1332] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:14:49.199216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d0d1-6665-491a-9f60-74e28927ffbd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:49.199862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d0d1-6665-491a-9f60-74e28927ffbd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:49.199862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d0d1-6665-491a-9f60-74e28927ffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:49.319370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:49.323943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d715d0d1-6665-491a-9f60-74e28927ffbd None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:49.326496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d715d0d1-6665-491a-9f60-74e28927ffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:49.478014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 675/1333] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:49.522342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:49.524079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:49.646486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd5d77-864d-441b-a0ca-d16e6e1e9c01 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:49.736584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:49.737275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:49.737654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:49.738099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:49.740327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:49.777099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:49.908966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:49.911875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b2d07-5e45-400e-8bb2-a5f6cb706a32 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:49.950556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 659/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:52.287242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:52.288058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:52.288282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:52.291002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 676/1335] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:52.298809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf32207-9915-42d5-8b82-6efd4bb1dffa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:52.299512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf32207-9915-42d5-8b82-6efd4bb1dffa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:52.299769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf32207-9915-42d5-8b82-6efd4bb1dffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:52.402278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4bf32207-9915-42d5-8b82-6efd4bb1dffa None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:52.403907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4bf32207-9915-42d5-8b82-6efd4bb1dffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:52.548931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:52.577393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 660/1336] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:52.621069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:52.623916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:52.798885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7838d9-3877-43c3-8885-bf9a5c56e877 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:52.844459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:52.902828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:52.903495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:52.903780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:52.904058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:52.905834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:53.043244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:53.044897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37eef3d-29ec-4d10-8b66-d988e5e46117 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:53.066837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 677/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:14:55.699918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:55.700606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:55.700606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:55.702075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 661/1338] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:14:55.709009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab37a0-cf2a-4c49-92c9-3c589127f837 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:55.709804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab37a0-cf2a-4c49-92c9-3c589127f837 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:55.709804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab37a0-cf2a-4c49-92c9-3c589127f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:55.793355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fbab37a0-cf2a-4c49-92c9-3c589127f837 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:14:55.794860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fbab37a0-cf2a-4c49-92c9-3c589127f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:55.805114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:55.923370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 678/1339] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:14:55.961314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:14:55.963318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:56.156659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba3b2-1fe3-43cc-9ff0-23b20c621b27 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:14:56.314292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:14:56.315333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:14:56.315821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:14:56.316308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:14:56.319800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:14:56.372160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:14:56.487110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:14:56.489194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827a865-a51d-40a7-a731-a08f51993647 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:14:56.523618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 662/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:01.556362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:01.557233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:01.557233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:01.559641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 679/1341] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:01.567712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ed6-abef-4a60-90e4-675d358acdaa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:01.568322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ed6-abef-4a60-90e4-675d358acdaa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:01.568322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ed6-abef-4a60-90e4-675d358acdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:01.680336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b9551ed6-abef-4a60-90e4-675d358acdaa None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:01.682021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b9551ed6-abef-4a60-90e4-675d358acdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:01.799812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:01.816555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 663/1342] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:15:01.856485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:01.858182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:02.012234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d5b0-e501-41ec-b161-f1f867a6ccb5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:02.073221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:02.120181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:02.120988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:02.121423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:02.122576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:02.123823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:02.261323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:02.262821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b50c3-ba9c-4e21-991d-b084e3c1d7cb service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:02.284951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 680/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:05.609832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:05.610650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:05.610650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:05.612388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 664/1344] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:05.620549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97eff-b710-4495-a7b0-42903bf5a89c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:05.621164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97eff-b710-4495-a7b0-42903bf5a89c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:05.621536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d97eff-b710-4495-a7b0-42903bf5a89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:05.754922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c9d97eff-b710-4495-a7b0-42903bf5a89c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:05.757062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c9d97eff-b710-4495-a7b0-42903bf5a89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:05.795354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:05.959595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 681/1345] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:06.023337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:06.024933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:06.180696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e24a7-0451-440e-8bec-1c4815a4ea85 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:06.300479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:06.301513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:06.301994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:06.302395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:06.305681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:06.384010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:06.496106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:06.497880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5890c94-6127-45d9-a8e5-6413ba9e2c4d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:06.522223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 665/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:12.284941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:12.285916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:12.285916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:12.288146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 682/1347] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:12.295870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846673f-5b8f-4cb5-9bf1-343020ea54ff None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:12.296576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846673f-5b8f-4cb5-9bf1-343020ea54ff None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:12.296923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846673f-5b8f-4cb5-9bf1-343020ea54ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:12.414085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8846673f-5b8f-4cb5-9bf1-343020ea54ff None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:12.415463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8846673f-5b8f-4cb5-9bf1-343020ea54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:12.580698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:12.586561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 666/1348] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:12.639229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:12.642229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:12.877797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5e0f5-deae-424a-a931-4bb3badbc2a2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:12.972961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:13.025202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:13.026253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:13.026721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:13.027175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:13.030576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:13.253281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:13.256435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6343b-6a41-4279-8156-c0b2e272aa26 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:13.294791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 683/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:20.128002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa286b-8155-42d5-9539-b0e7110b3c48 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:20.128741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa286b-8155-42d5-9539-b0e7110b3c48 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:20.129096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa286b-8155-42d5-9539-b0e7110b3c48 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:20.136104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 667/1350] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:15:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:20.143040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0181-ded0-45ba-ac97-2fc599c6ca9f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:20.143848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0181-ded0-45ba-ac97-2fc599c6ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:20.143848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d0181-ded0-45ba-ac97-2fc599c6ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:20.333724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-378d0181-ded0-45ba-ac97-2fc599c6ca9f None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:20.335843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-378d0181-ded0-45ba-ac97-2fc599c6ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:20.380672 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:20.570085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 684/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:15:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:26.391509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb89dd-3a60-4c12-a3fa-6d06ebfec948 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:26.392664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb89dd-3a60-4c12-a3fa-6d06ebfec948 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:26.393185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb89dd-3a60-4c12-a3fa-6d06ebfec948 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:26.395060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 668/1352] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:15:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:26.401958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7b60-9419-4218-9b89-11cb6c7890f3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:26.402601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7b60-9419-4218-9b89-11cb6c7890f3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:26.403310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b7b60-9419-4218-9b89-11cb6c7890f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:26.555851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ba9b7b60-9419-4218-9b89-11cb6c7890f3 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:26.558015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ba9b7b60-9419-4218-9b89-11cb6c7890f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:26.727903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:26.815441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 685/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:28.862340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3550-c579-4c32-95e2-1a4c863f2f05 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:28.863264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3550-c579-4c32-95e2-1a4c863f2f05 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:28.863264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3550-c579-4c32-95e2-1a4c863f2f05 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:28.864343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 669/1354] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:28.871572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff6d-56b3-4610-b24e-2743ff40ec20 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:28.872464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff6d-56b3-4610-b24e-2743ff40ec20 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:28.872864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff6d-56b3-4610-b24e-2743ff40ec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:28.982630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-090eff6d-56b3-4610-b24e-2743ff40ec20 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:28.983910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-090eff6d-56b3-4610-b24e-2743ff40ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:29.138667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 686/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:33.015599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319cfe-b1fb-4462-baff-9b54f0ba4717 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:33.016392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319cfe-b1fb-4462-baff-9b54f0ba4717 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:33.016392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319cfe-b1fb-4462-baff-9b54f0ba4717 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:33.017828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 670/1356] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:15:33.027662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c65a-0cac-412d-9a05-c000d9331167 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:33.028320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c65a-0cac-412d-9a05-c000d9331167 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:33.028650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c65a-0cac-412d-9a05-c000d9331167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:33.108568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:33.152643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-63b5c65a-0cac-412d-9a05-c000d9331167 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:33.154152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-63b5c65a-0cac-412d-9a05-c000d9331167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:33.377168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 687/1357] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:33.463803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:33.464818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:33.465728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:33.468532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 671/1358] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 25 16:15:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:15:33.487806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5dba4-331a-478c-9efd-c41b4fc6b5b1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:33.489146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5dba4-331a-478c-9efd-c41b4fc6b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:33.489908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5dba4-331a-478c-9efd-c41b4fc6b5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:33.637545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4ed5dba4-331a-478c-9efd-c41b4fc6b5b1 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:33.639866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4ed5dba4-331a-478c-9efd-c41b4fc6b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:33.746237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:33.861231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 688/1359] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 25 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:15:33.874756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:33.877493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:34.110286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b21a-1fdb-467e-8c2a-8a876ec36a41 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:34.245595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:34.306915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:34.307692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:34.308041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:34.308384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:34.310725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:34.499435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:34.503377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15caed-f6f2-4c62-a6d0-54b17c9a5afb service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:34.544896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 672/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:15:37.526417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:37.527732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:37.528378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:37.531378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 689/1361] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:15:37.541910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2659f-7bb3-4a34-90b5-bad9384ea7a0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:37.542739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2659f-7bb3-4a34-90b5-bad9384ea7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:37.543028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2659f-7bb3-4a34-90b5-bad9384ea7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:37.691301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-89b2659f-7bb3-4a34-90b5-bad9384ea7a0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:37.692979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-89b2659f-7bb3-4a34-90b5-bad9384ea7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:37.755094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:37.881395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 673/1362] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:37.937863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:37.940889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:38.268121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6704710-a61e-499f-ac80-f3ab7620de45 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:38.296888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:38.483350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:38.484468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:38.484987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:38.485448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:38.488743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:38.639388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:38.641949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98815e-bf20-4d85-b3e0-d1d91dd7c7a5 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:38.670288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 690/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:41.899680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020ff40-3180-46d7-81a0-c1920a51ed04 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:41.900539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020ff40-3180-46d7-81a0-c1920a51ed04 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:41.900539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020ff40-3180-46d7-81a0-c1920a51ed04 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:41.902211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 674/1364] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:41.908633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a671f3-2b3d-4b27-9930-bf9ed1542520 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:41.909642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a671f3-2b3d-4b27-9930-bf9ed1542520 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:41.909642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a671f3-2b3d-4b27-9930-bf9ed1542520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:41.932928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:42.015899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-59a671f3-2b3d-4b27-9930-bf9ed1542520 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:42.017221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-59a671f3-2b3d-4b27-9930-bf9ed1542520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:42.162282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 691/1365] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:15:42.214933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:42.215303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:42.215586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:42.217360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 675/1366] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 25 16:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:42.223554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118437fe-f487-4507-be2e-1aeb80ac2f65 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:42.223945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118437fe-f487-4507-be2e-1aeb80ac2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:42.224173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118437fe-f487-4507-be2e-1aeb80ac2f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:42.324821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-118437fe-f487-4507-be2e-1aeb80ac2f65 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:42.326176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-118437fe-f487-4507-be2e-1aeb80ac2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:42.371357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:42.479636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 692/1367] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 25 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:42.483774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:42.485420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:42.708728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b65557-0721-4395-a2b1-0fe19ee96cdc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:42.757304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:42.961480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:42.962534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:42.962957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:42.963447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:42.966871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:43.157458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:43.159247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37b1de-ecb4-4921-9eb3-d4b3bc0c7fbc service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:43.184552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 676/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:15:44.342481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae19a8b-4e79-4d3b-898d-907efbd223dd None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:44.343390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae19a8b-4e79-4d3b-898d-907efbd223dd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:44.343390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae19a8b-4e79-4d3b-898d-907efbd223dd None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:44.345081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 693/1369] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:15:44.354928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173a335-83de-45df-9d3e-6aab33a4c908 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:44.355575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173a335-83de-45df-9d3e-6aab33a4c908 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:44.355927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173a335-83de-45df-9d3e-6aab33a4c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:44.407123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:44.473940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a173a335-83de-45df-9d3e-6aab33a4c908 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:44.476221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a173a335-83de-45df-9d3e-6aab33a4c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:44.693591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 677/1370] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:45.233958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:45.234482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:45.234974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:45.237686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 694/1371] 10.222.0.66 () {54 vars in 964 bytes} [Mon Jul 25 16:15:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:45.247773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd388-4e7d-4767-8e51-eb34356272cb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:45.247773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd388-4e7d-4767-8e51-eb34356272cb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:45.248131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd388-4e7d-4767-8e51-eb34356272cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:45.393480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-46abd388-4e7d-4767-8e51-eb34356272cb None None] MFA Rules not processed for user `468fc86f361a4c7c91cacc1d90086176`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:45.395901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-46abd388-4e7d-4767-8e51-eb34356272cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:45.504670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:45.585999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 678/1372] 10.222.0.66 () {58 vars in 1061 bytes} [Mon Jul 25 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:45.599797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:45.602663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:45.835836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992448f9-a869-47cf-806a-7be1709c81f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:45.923474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:46.009223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:46.010361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:46.010905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:46.011346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:46.014617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:46.307219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:46.310181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bf0b-9be5-4d35-81ab-6bb2703181ed service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:46.365962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 695/1373] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:46.392924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:46.393136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:46.394206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:46.397696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 679/1374] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:46.406533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aba3c-c219-48da-be97-9b836b3cad43 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:46.406997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aba3c-c219-48da-be97-9b836b3cad43 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:46.407448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aba3c-c219-48da-be97-9b836b3cad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:46.536142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-297aba3c-c219-48da-be97-9b836b3cad43 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:46.537540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-297aba3c-c219-48da-be97-9b836b3cad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:46.566731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:46.724646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 696/1375] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:46.807941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:46.811168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:47.127284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8713b-d408-488a-b18a-179d812af10d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:47.217467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:47.306921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:47.307711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:47.308131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:47.308594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:47.310772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:47.472529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:47.474441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840745b-7cb4-47c3-a80a-c757dc0cae57 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:47.500335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 680/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:50.931800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:50.932666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:50.932951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:50.934646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 697/1377] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:50.943362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8806b-67ca-473b-8962-3185a7847710 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:50.944088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8806b-67ca-473b-8962-3185a7847710 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:50.944328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8806b-67ca-473b-8962-3185a7847710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:51.077486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fbf8806b-67ca-473b-8962-3185a7847710 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:51.079365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fbf8806b-67ca-473b-8962-3185a7847710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:51.090997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:51.293998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 681/1378] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:51.362334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:51.364372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:51.660016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c6b94-f400-4012-8b9d-825a18e486ad None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:51.671238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:51.827051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:51.827917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:51.828218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:51.828502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:51.830817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:51.964296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:51.966545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abce39-6aba-4b34-9831-25073e467888 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:52.002355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 698/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:55.038148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:55.039070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:55.039070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:55.040161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 682/1380] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:15:55.046600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3b002-049f-41ed-8cce-75f700812ee8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:55.047540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3b002-049f-41ed-8cce-75f700812ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:55.047540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3b002-049f-41ed-8cce-75f700812ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:55.061514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:55.150334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ecf3b002-049f-41ed-8cce-75f700812ee8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:55.151838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ecf3b002-049f-41ed-8cce-75f700812ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:55.327213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 699/1381] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:15:55.383470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:55.386634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:55.617776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:55.636117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76d817-5804-4823-bf83-d6fb97dd1fcf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:55.839642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:55.840473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:55.840781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:55.841067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:55.843180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:56.022459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:56.024553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed755e-4fb8-4450-8d34-58ea77082253 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:56.038836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:56.051763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 683/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:15:58.887906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:58.888585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:58.888910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:58.891825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 700/1383] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:15:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:15:58.899275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dfa3-1f0d-4100-ad2f-3cbc97d88963 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:58.900174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dfa3-1f0d-4100-ad2f-3cbc97d88963 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:58.900174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dfa3-1f0d-4100-ad2f-3cbc97d88963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:59.023009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ae19dfa3-1f0d-4100-ad2f-3cbc97d88963 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:15:59.026183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ae19dfa3-1f0d-4100-ad2f-3cbc97d88963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:59.223255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 684/1384] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:15:59.281303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:15:59.283469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:59.422937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:59.573919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823339-83a1-4a54-b5a1-681ce78c2e55 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:15:59.773032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:15:59.773814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:15:59.774105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:15:59.774388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:15:59.776363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:15:59.845439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:15:59.907655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:15:59.909638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2601c-c9d6-4acc-a293-7f093990047d service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:15:59.944057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 701/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:02.702673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:02.703298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:02.703568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:02.706119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 685/1386] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:02.715352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed662d3-00aa-43bd-88ae-5a08becf352a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:02.715987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed662d3-00aa-43bd-88ae-5a08becf352a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:02.716355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed662d3-00aa-43bd-88ae-5a08becf352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:02.844588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eed662d3-00aa-43bd-88ae-5a08becf352a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:02.846047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eed662d3-00aa-43bd-88ae-5a08becf352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:03.011268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 702/1387] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:03.093954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:03.096819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:03.227601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:03.363518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd89e-25ff-4d96-a859-f302e7f8ba09 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:03.585898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:03.587033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:03.587535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:03.588035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:03.591585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:03.669020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:03.780953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:03.784228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1940015-22fc-4e23-855b-f6675c8fcf34 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:03.823313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 686/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:05.100765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909213-c225-459b-a910-10285bab40ca None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:05.101571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909213-c225-459b-a910-10285bab40ca None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:05.101799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909213-c225-459b-a910-10285bab40ca None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:05.104798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 703/1389] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:05.113565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100330a-8618-4444-b433-12e0f255e84d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:05.114490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100330a-8618-4444-b433-12e0f255e84d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:05.114490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100330a-8618-4444-b433-12e0f255e84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:05.244165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9100330a-8618-4444-b433-12e0f255e84d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:05.246435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9100330a-8618-4444-b433-12e0f255e84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:05.439829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 687/1390] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:05.538561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:05.539064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:05.539568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:05.542190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 704/1391] 10.222.0.66 () {54 vars in 966 bytes} [Mon Jul 25 16:16:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:16:05.551564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ebde2-f641-4f95-8c80-9c20d8a6cab6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:05.552054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ebde2-f641-4f95-8c80-9c20d8a6cab6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:05.552548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ebde2-f641-4f95-8c80-9c20d8a6cab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:05.585108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:05.710889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8f9ebde2-f641-4f95-8c80-9c20d8a6cab6 None None] MFA Rules not processed for user `0c159db46d48429b95e0166596c53d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:05.712972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8f9ebde2-f641-4f95-8c80-9c20d8a6cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:05.877737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 688/1392] 10.222.0.66 () {58 vars in 1063 bytes} [Mon Jul 25 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:05.891215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:05.894230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:05.957316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:06.092464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24ea3e-66bd-4af7-936a-1c92869504cd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:06.232920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:06.234098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:06.234579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:06.235062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:06.238423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:06.387816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:06.448698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:06.451627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f9fc-701c-479c-981e-3a716e3c057b service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:06.486346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 705/1393] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:06.731626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:06.732232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:06.732664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:06.735620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 689/1394] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:06.744803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad5bf-a446-491e-8403-10a12dbc3e25 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:06.745250 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad5bf-a446-491e-8403-10a12dbc3e25 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:06.745725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad5bf-a446-491e-8403-10a12dbc3e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:06.881249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f85ad5bf-a446-491e-8403-10a12dbc3e25 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:06.883633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f85ad5bf-a446-491e-8403-10a12dbc3e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:07.069537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 706/1395] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:07.141668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:07.144530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:07.306468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:07.367383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467622b9-2056-4fbb-b822-8694fffa347f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:07.570232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800db0c-6bca-4d55-9db7-1f26daf2e46d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:07.570927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800db0c-6bca-4d55-9db7-1f26daf2e46d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:07.571104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800db0c-6bca-4d55-9db7-1f26daf2e46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:07.591428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:07.592502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:07.593018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:07.593493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:07.595960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:07.650840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:07.696707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1800db0c-6bca-4d55-9db7-1f26daf2e46d None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:07.698223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1800db0c-6bca-4d55-9db7-1f26daf2e46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:07.708741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:07.753095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:07.756418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe1f1d-7937-4be1-a240-e52031041f3c service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:07.786885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 690/1396] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:07.833252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 707/1397] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:07.899285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:07.899909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:07.900231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:07.902625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 691/1398] 10.222.0.66 () {54 vars in 943 bytes} [Mon Jul 25 16:16:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:07.912061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34316-3fa2-45e4-b06c-aa08c3996164 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:07.912594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34316-3fa2-45e4-b06c-aa08c3996164 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:07.913065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34316-3fa2-45e4-b06c-aa08c3996164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:08.064412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-41f34316-3fa2-45e4-b06c-aa08c3996164 None None] MFA Rules not processed for user `4f36fd1f37394b49ac38568200578c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:08.067178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-41f34316-3fa2-45e4-b06c-aa08c3996164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:08.167814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:08.236513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 708/1399] 10.222.0.66 () {58 vars in 1040 bytes} [Mon Jul 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:16:08.252714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:08.255930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:08.467548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1403-0df2-467e-82f4-d238a86921dd None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:08.554420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:08.652390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:08.653548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:08.654062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:08.654548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:08.658037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:08.808072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:08.810568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce50b-2c74-4f96-907a-62f65ee83d08 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:08.821756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 692/1400] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:16:10.669965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:10.670790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:10.670975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:10.681058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 709/1401] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:10.684068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a66a6-cede-408e-95b2-f9005e096918 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:10.684902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a66a6-cede-408e-95b2-f9005e096918 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:10.685584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a66a6-cede-408e-95b2-f9005e096918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:10.834355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2c8a66a6-cede-408e-95b2-f9005e096918 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:10.836551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2c8a66a6-cede-408e-95b2-f9005e096918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:10.932799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:11.067814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 693/1402] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:11.155148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:11.157980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:11.490816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ade09-40e9-4dbd-8c33-e5596de04259 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:11.584206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:11.753429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:11.753429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:11.753429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:11.754854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:11.757077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:11.944688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:11.948257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296db622-6f80-41d0-a805-096c35ae6b92 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:11.988329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 710/1403] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:14.822353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:14.823182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:14.823182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:14.827025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 694/1404] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:14.833895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93a06-4d14-47a5-b0fb-7d3903edd550 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:14.834504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93a06-4d14-47a5-b0fb-7d3903edd550 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:14.834504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93a06-4d14-47a5-b0fb-7d3903edd550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:14.933885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:14.949389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f9e93a06-4d14-47a5-b0fb-7d3903edd550 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:14.950660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f9e93a06-4d14-47a5-b0fb-7d3903edd550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:15.115967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 711/1405] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:15.185576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:15.187679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:15.450498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9084f53-8d96-4183-accb-ff52c51cf901 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:15.506473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:15.737934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:15.739452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:15.740100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:15.740724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:15.744243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:15.937728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:15.941102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a12a54-7940-4a8a-9d9d-6e6fc5658a67 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:15.985524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 695/1406] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:19.513110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:19.514878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:19.515382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:19.523874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 712/1407] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:16:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:19.532231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0314aa-35a3-44bc-9cb1-1bcffb7d7d06 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:19.532871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0314aa-35a3-44bc-9cb1-1bcffb7d7d06 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:19.533226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0314aa-35a3-44bc-9cb1-1bcffb7d7d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:19.620343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:19.687828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-aa0314aa-35a3-44bc-9cb1-1bcffb7d7d06 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:19.689463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aa0314aa-35a3-44bc-9cb1-1bcffb7d7d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:19.883904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 696/1408] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:19.958043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:19.960922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:20.187967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:20.226015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062b17-3dc1-496f-a828-f631cc5fdfa4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:20.528212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:20.528212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:20.528212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:20.528212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:20.532734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:20.697051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:20.704270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:20.707415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c17d6-2b09-4945-9d96-c6cc41e8c39d service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:20.738989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 713/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:29.879371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=52 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:30.734406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:39.612773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2ac4-a71a-4757-b707-2aa31bac6254 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:39.613448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2ac4-a71a-4757-b707-2aa31bac6254 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:39.613448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2ac4-a71a-4757-b707-2aa31bac6254 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:39.614806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 697/1410] 10.222.0.66 () {54 vars in 918 bytes} [Mon Jul 25 16:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:39.621463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae67389-191a-40c0-b9fc-1e7d996c128d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:39.622389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae67389-191a-40c0-b9fc-1e7d996c128d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:39.622389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae67389-191a-40c0-b9fc-1e7d996c128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:39.740681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7ae67389-191a-40c0-b9fc-1e7d996c128d None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:39.742054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7ae67389-191a-40c0-b9fc-1e7d996c128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:39.874622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 714/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 25 16:16:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:39.891461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d940e8-29e8-414d-adcf-cd6eb663fe4c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:39.891749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d940e8-29e8-414d-adcf-cd6eb663fe4c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:39.892026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d940e8-29e8-414d-adcf-cd6eb663fe4c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:39.893667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 698/1412] 10.222.0.66 () {54 vars in 918 bytes} [Mon Jul 25 16:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:39.899973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec3a85-40bb-4e3f-afd2-2b303cda79c4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:39.900253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec3a85-40bb-4e3f-afd2-2b303cda79c4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:39.900552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec3a85-40bb-4e3f-afd2-2b303cda79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:39.998494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:40.005199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-50ec3a85-40bb-4e3f-afd2-2b303cda79c4 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:40.007605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-50ec3a85-40bb-4e3f-afd2-2b303cda79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:40.212364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 715/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 25 16:16:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:43.162414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:43.163247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:43.163247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:43.165480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:43.175182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcced15b-c950-4252-b7b1-9ac511f2b1d5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:43.176056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcced15b-c950-4252-b7b1-9ac511f2b1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:43.176056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcced15b-c950-4252-b7b1-9ac511f2b1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:43.307677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dcced15b-c950-4252-b7b1-9ac511f2b1d5 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:43.310123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcced15b-c950-4252-b7b1-9ac511f2b1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:43.433535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:43.516546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:43.590159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:43.592044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:43.790978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ecfb1-3ab5-4b96-a8ec-fae6baa4ceac None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:43.894950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:43.957462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:43.958139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:43.958425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:43.958691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:43.960491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:44.092740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:44.095019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba905882-05d4-4763-9dfe-d980e7bf6cfb service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:44.117404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 700/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:45.440054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:45.440722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:45.440722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:45.442845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:45.450359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c0bf7-9d39-42d2-a5dc-74bae077029d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:45.451020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c0bf7-9d39-42d2-a5dc-74bae077029d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:45.451292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c0bf7-9d39-42d2-a5dc-74bae077029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:45.556981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-846c0bf7-9d39-42d2-a5dc-74bae077029d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:45.558339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-846c0bf7-9d39-42d2-a5dc-74bae077029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:45.591680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:45.719412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:45.892194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:45.895134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:46.121914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d650ba-4b11-46ae-b0bc-d0aa51773171 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:46.182167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:46.293177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:46.293954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:46.294276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:46.294639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:46.296715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:46.598137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:46.599829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b80679-2582-4388-827d-4fbecc2fc892 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:46.623344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 718/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:47.618993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:47.619886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:47.619886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:47.621596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:47.628111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d0ed4-94e8-4b79-afc1-995685205852 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:47.628682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d0ed4-94e8-4b79-afc1-995685205852 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:47.628682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d0ed4-94e8-4b79-afc1-995685205852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:47.717148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:47.751448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5a4d0ed4-94e8-4b79-afc1-995685205852 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:47.753000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a4d0ed4-94e8-4b79-afc1-995685205852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:47.899601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:48.078280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:48.079912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:48.319011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717a5e0-cd80-4803-a9d4-05e0a3cfcf12 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:48.344429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:48.521500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:48.522658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:48.523154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:48.523674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:48.527054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:48.708525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:48.711497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9e6b8-f63e-4cfc-a35f-eeb4cbdf1373 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:48.739729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 703/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:49.774177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:49.775019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:49.775019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:49.776389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:49.784305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e54-0020-46ba-81d0-04f3e8723a97 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:49.785218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e54-0020-46ba-81d0-04f3e8723a97 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:49.785218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e54-0020-46ba-81d0-04f3e8723a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:49.815161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:49.916153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d2217e54-0020-46ba-81d0-04f3e8723a97 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:49.917998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2217e54-0020-46ba-81d0-04f3e8723a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:50.059284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:16:50.221260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:50.223126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:50.378257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:50.446155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e74d4-a6af-4f85-bb00-dd4256b7660d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:50.685876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:50.687002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:50.687507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:50.687987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:50.691578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:50.847231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:50.870435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:50.872765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b84bce-4a0f-4f73-93a4-34949930a5e9 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:50.910990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 721/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:52.156715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:52.157425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:52.157425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:52.159398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:52.166070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3a4e0-f0b6-4ba9-bf17-7d9e343b4266 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:52.167003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3a4e0-f0b6-4ba9-bf17-7d9e343b4266 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:52.167003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3a4e0-f0b6-4ba9-bf17-7d9e343b4266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:52.264924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-09c3a4e0-f0b6-4ba9-bf17-7d9e343b4266 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:52.266314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-09c3a4e0-f0b6-4ba9-bf17-7d9e343b4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:52.384864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:52.555058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:52.556679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:52.668682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:52.745466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511c7d6-b1a1-4105-9c4f-b84437897f4c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:52.916446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:52.917268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:52.917588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:52.917883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:52.920654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:52.995220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:53.060841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:53.064081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70603139-6c23-44b5-90e9-00320c283911 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:53.099064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 706/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:53.990054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:53.990684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:53.990981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:53.993845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:54.000117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0605-aae4-4944-a233-da1c61d5e35c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:54.000787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0605-aae4-4944-a233-da1c61d5e35c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:54.001022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0605-aae4-4944-a233-da1c61d5e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:54.106721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a9dd0605-aae4-4944-a233-da1c61d5e35c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:54.107984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a9dd0605-aae4-4944-a233-da1c61d5e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:54.285071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:54.530679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:54.532401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:54.605415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:54.739388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0a6b-5b52-4c13-9f33-148251255c0b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:54.905368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:54.905950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:54.906227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:54.906500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:54.908348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:54.916874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:55.029191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:55.031000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3de57-35f6-4281-97d8-ded7dba0209e service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:55.054047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 724/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:55.933289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:55.934090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:55.934090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:55.935604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:55.941758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ccf9-396d-4d1d-9738-221c00dfc13f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:55.942102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ccf9-396d-4d1d-9738-221c00dfc13f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:55.942414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ccf9-396d-4d1d-9738-221c00dfc13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:56.032129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dc08ccf9-396d-4d1d-9738-221c00dfc13f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:56.033527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc08ccf9-396d-4d1d-9738-221c00dfc13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:56.119880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:56.156642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:56.314060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:56.315781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:56.338612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:56.514450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739ed8c-c147-457a-a06f-542f4be13688 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:56.631856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:56.668248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:56.669081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:56.669561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:56.669945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:56.671948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:56.815962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:56.817847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896aa749-e98f-4741-9384-94a1abd28d3a service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:56.842970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 709/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:16:57.755615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:57.756283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:57.756283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:57.757933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:57.764321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee1df-48ae-4b0f-aa26-c61d7dfedefd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:57.764977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee1df-48ae-4b0f-aa26-c61d7dfedefd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:57.764977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee1df-48ae-4b0f-aa26-c61d7dfedefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:57.874499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-174ee1df-48ae-4b0f-aa26-c61d7dfedefd None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:57.876609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-174ee1df-48ae-4b0f-aa26-c61d7dfedefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:57.932688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:58.006040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:16:58.167356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:58.169311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:58.368213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc305f09-6667-4e79-864b-3cab05ae2cf3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:16:58.447053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:58.525182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:16:58.525908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:58.526195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:58.526465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:58.528293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:58.647360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:16:58.649099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281813e-acf0-4303-b760-6661a2b7f5bd service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:16:58.672765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 727/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:16:59.600107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:59.600752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:59.600752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:59.602183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:16:59.607963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0a5a-2133-4143-93c3-d91d5a80be56 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:16:59.608538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0a5a-2133-4143-93c3-d91d5a80be56 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:16:59.608538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0a5a-2133-4143-93c3-d91d5a80be56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:16:59.697566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e96f0a5a-2133-4143-93c3-d91d5a80be56 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:16:59.698858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e96f0a5a-2133-4143-93c3-d91d5a80be56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:16:59.701212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:16:59.824070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:16:59.989137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:16:59.990732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:00.212917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66257e-eacf-410a-bfcf-0c28c5ecdb5e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:00.251283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:00.364516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:00.365380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:00.365783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:00.366152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:00.368015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:00.484177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:00.486092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecdfb1-28b8-4049-8f8e-44dbdf921dc7 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:00.510155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 712/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:03.214125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:03.214777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:03.215042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:03.216731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:03.225961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17932e-a355-481c-ba01-fe11539fb2fc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:03.227083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17932e-a355-481c-ba01-fe11539fb2fc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:03.227083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17932e-a355-481c-ba01-fe11539fb2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:03.309192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:03.360657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ea17932e-a355-481c-ba01-fe11539fb2fc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:03.362278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ea17932e-a355-481c-ba01-fe11539fb2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:03.504181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:03.552938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:03.554713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:03.748303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83a31-88ee-4b7a-a86a-19dab9d0ca86 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:03.791658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:03.849089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:03.850265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:03.850265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:03.850595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:03.853098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:04.062527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:04.065221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b76d3d-6958-4b3f-a4b6-c34ad6c5d65c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:04.112740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 730/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:07.119484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:07.120415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:07.120415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:07.122102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:17:07.129086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadba57-48b8-40fe-814c-3520528654e1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:07.130103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadba57-48b8-40fe-814c-3520528654e1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:07.130252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadba57-48b8-40fe-814c-3520528654e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:07.262707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5dadba57-48b8-40fe-814c-3520528654e1 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:07.264484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5dadba57-48b8-40fe-814c-3520528654e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:07.284692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:07.431174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:17:07.475858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:07.477989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:07.624796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819549d-15c5-45b3-9344-f956e422da61 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:07.651275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:07.783198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:07.784355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:07.784831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:07.785307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:07.788676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:08.000637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:08.003615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1102e6-448f-4475-86e1-7f852b9d1624 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:08.042169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 715/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:11.089946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:11.090837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:11.091020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:11.093661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:11.105041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f16a6-dd05-4ace-8585-acbe7b42d1be None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:11.106131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f16a6-dd05-4ace-8585-acbe7b42d1be None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:11.106765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f16a6-dd05-4ace-8585-acbe7b42d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:11.192800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:11.220852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-720f16a6-dd05-4ace-8585-acbe7b42d1be None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:11.222434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-720f16a6-dd05-4ace-8585-acbe7b42d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:11.378460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:11.428259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:11.430781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:11.590546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5ee1-4089-4c5e-9884-2354c33f1259 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:11.650567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:11.711446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:11.712577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:11.713067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:11.713600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:11.716971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:11.916907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:11.918901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934d776-2985-4465-9d4d-4667164c7f42 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:11.947073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 733/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:15.620199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:15.621092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:15.621472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:15.624374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:17:15.635303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedb3de-b873-4224-ae3d-d07ec65a79fe None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:15.636149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedb3de-b873-4224-ae3d-d07ec65a79fe None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:15.636417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedb3de-b873-4224-ae3d-d07ec65a79fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:15.757677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cfedb3de-b873-4224-ae3d-d07ec65a79fe None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:15.759052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cfedb3de-b873-4224-ae3d-d07ec65a79fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:15.793651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:15.944760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:16.004747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:16.007855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:16.191359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ddcbb-1451-42a2-b755-2a150f6b178c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:16.226576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:16.354711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:16.355488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:16.355815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:16.356102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:16.358590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:16.543178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:16.546128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaa1fc-1328-4621-a9ea-86d2a1b434b0 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:16.584406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 718/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:19.142706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:19.143288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:19.143288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:19.144807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:19.151475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11938-a537-4288-96bf-251161294426 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:19.152097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11938-a537-4288-96bf-251161294426 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:19.152097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11938-a537-4288-96bf-251161294426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:19.235344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:19.239341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-63c11938-a537-4288-96bf-251161294426 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:19.241929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-63c11938-a537-4288-96bf-251161294426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:19.392172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:19.448981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:19.451515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:19.616960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05a7e8-8cf2-48cc-8367-5c847679b942 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:19.680318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:19.714526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:19.715371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:19.715690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:19.715996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:19.718025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:19.897671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:19.900465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cd370-9136-4033-8155-628dbcaded87 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:19.936913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 736/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:22.569181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:22.570139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:22.570318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:22.572655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:22.581073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911ec7-bce3-4964-8f28-958085d0dd01 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:22.581834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911ec7-bce3-4964-8f28-958085d0dd01 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:22.581834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911ec7-bce3-4964-8f28-958085d0dd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:22.681009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2a911ec7-bce3-4964-8f28-958085d0dd01 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:22.682244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2a911ec7-bce3-4964-8f28-958085d0dd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:22.722334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:22.816047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:22.856891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:22.858734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:22.991876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79302e-f5ac-47b2-a5a9-2c259fd4ae84 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:23.029055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:23.087826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:23.088510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:23.088788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:23.089058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:23.090829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:23.239707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:23.241666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee41cf4-4057-4d89-b58a-d3b538dfaf87 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:23.278269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 721/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:26.376516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:26.377472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:26.377653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:26.380188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:26.390853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6af7-54b5-4ec8-9c5b-70d6d855961f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:26.391741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6af7-54b5-4ec8-9c5b-70d6d855961f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:26.391741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6af7-54b5-4ec8-9c5b-70d6d855961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:26.509072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-096d6af7-54b5-4ec8-9c5b-70d6d855961f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:26.510661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-096d6af7-54b5-4ec8-9c5b-70d6d855961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:26.522579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:26.679506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:26.746375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:26.750244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:26.970896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fe67c-8e3a-42c6-b1ef-c8fbe9d9d600 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:27.089816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:27.130229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:27.131420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:27.131979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:27.132489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:27.135999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:27.377649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:27.380493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa95ca-edf7-433b-b2fc-d88dac107278 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:27.420409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 739/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:29.935743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:29.936647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:29.936647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:29.938725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:29.945835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300212-730c-4619-9c14-c4ebf831cc21 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:29.946768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300212-730c-4619-9c14-c4ebf831cc21 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:29.946768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300212-730c-4619-9c14-c4ebf831cc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:30.055588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8e300212-730c-4619-9c14-c4ebf831cc21 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:30.056925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8e300212-730c-4619-9c14-c4ebf831cc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:30.133944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:30.266304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:30.336154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:30.339229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:30.560034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663fc6e-e445-47af-992f-795ef8bdbf8b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:30.643275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:30.719993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:30.721201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:30.721760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:30.722271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:30.725808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:30.939181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:30.941978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9853441-8e16-4459-bd16-49d0e589b44d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:30.971640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 724/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:33.680887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:33.681834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:33.681986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:33.684738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:33.694794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc01a-3edc-48a0-8f67-89beec51735b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:33.695644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc01a-3edc-48a0-8f67-89beec51735b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:33.695644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acc01a-3edc-48a0-8f67-89beec51735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:33.823259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-27acc01a-3edc-48a0-8f67-89beec51735b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:33.825489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-27acc01a-3edc-48a0-8f67-89beec51735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:33.860722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:34.041170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:17:34.082941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:34.085929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:34.297756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60213-8c88-4aff-878a-4ee0762be562 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:34.431132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:34.446513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:34.447676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:34.448156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:34.448629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:34.452093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:34.644025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:34.645837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c057dab-2766-4c11-86c7-a627fc1df1ab service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:34.671700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 742/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:36.978706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:36.979337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:36.979337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:36.980751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:17:36.988837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3aa73-5480-44ff-b0bb-63a3589a9d4a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:36.989608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3aa73-5480-44ff-b0bb-63a3589a9d4a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:36.989860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3aa73-5480-44ff-b0bb-63a3589a9d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:37.128695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-05c3aa73-5480-44ff-b0bb-63a3589a9d4a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:17:37.130197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-05c3aa73-5480-44ff-b0bb-63a3589a9d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:37.203142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:37.293160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:17:37.348202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:17:37.350088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:37.550625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41068-558e-4d9a-9c2f-b3f444c7ce29 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:17:37.651354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:37.707780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:17:37.709042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:17:37.709808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:17:37.710309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:17:37.713806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:17:37.950811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:17:37.953798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e465ef4-b5d9-430c-8d5f-14a723588d7f service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:17:37.993663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 727/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:17:47.290129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=22 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:17:47.988147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=66 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:06.910551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c770c1-1ce7-436f-a2d8-8231df31847c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:06.911507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c770c1-1ce7-436f-a2d8-8231df31847c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:06.911507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c770c1-1ce7-436f-a2d8-8231df31847c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:06.912564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 744/1471] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:18:06.918964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754dfe-c61c-41ff-bb31-572e4ee2d859 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:06.919631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754dfe-c61c-41ff-bb31-572e4ee2d859 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:06.919631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a754dfe-c61c-41ff-bb31-572e4ee2d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:07.011645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6a754dfe-c61c-41ff-bb31-572e4ee2d859 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:07.012925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6a754dfe-c61c-41ff-bb31-572e4ee2d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:07.148043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 728/1472] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:07.254954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:07.255352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:07.255842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:07.258142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 745/1473] 10.222.0.66 () {54 vars in 966 bytes} [Mon Jul 25 16:18:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:18:07.265909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770a171-3a2e-4ea4-97fc-3209b2444836 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:07.266702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770a171-3a2e-4ea4-97fc-3209b2444836 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:07.267201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770a171-3a2e-4ea4-97fc-3209b2444836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:07.391933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9770a171-3a2e-4ea4-97fc-3209b2444836 None None] MFA Rules not processed for user `0c159db46d48429b95e0166596c53d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:07.393311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9770a171-3a2e-4ea4-97fc-3209b2444836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:07.430209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:07.500761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 729/1474] 10.222.0.66 () {58 vars in 1063 bytes} [Mon Jul 25 16:18:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:07.508894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:07.510562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:07.647435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3457d-b695-461f-ad57-2bb82d6b77ce None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:07.737699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:07.738543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:07.738931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:07.739249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:07.741223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:07.816472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:07.888419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:07.890084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3845b9-9092-4e66-b100-bb1b4b0a2cd5 service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:07.915379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 746/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:18:10.357282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:10.358259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:10.358259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:10.359876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 730/1476] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:18:10.368587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884294fd-a2b1-4753-8f55-a8ed2ec5d5f3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:10.369314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884294fd-a2b1-4753-8f55-a8ed2ec5d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:10.369598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884294fd-a2b1-4753-8f55-a8ed2ec5d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:10.464678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-884294fd-a2b1-4753-8f55-a8ed2ec5d5f3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:10.466143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-884294fd-a2b1-4753-8f55-a8ed2ec5d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:10.590478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 747/1477] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:10.617998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:10.617998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:10.769639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ded9b-0cd8-4fc1-88b1-a3a5f20468b4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:10.834178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:10.894952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:10.895694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:10.896041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:10.896357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:10.897174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:10.899229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:18:10.899229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:18:10.899229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:18:10.900670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb19feb-d01d-4a4f-bf98-f8e72e8fa477 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:10.926957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 731/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 25 16:18:10 2022] GET /identity/v3/projects => generated 2553 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:18:13.540630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:13.541770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:13.541770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:13.542709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 748/1479] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:18:13.550527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31027c7-d756-485d-87f7-5e0af0ef2284 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:13.551172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31027c7-d756-485d-87f7-5e0af0ef2284 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:13.551518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31027c7-d756-485d-87f7-5e0af0ef2284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:13.656367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d31027c7-d756-485d-87f7-5e0af0ef2284 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:13.658472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d31027c7-d756-485d-87f7-5e0af0ef2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:13.789072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 732/1480] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:13.842322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:13.844902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:13.857584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:14.035394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa2c6f-9c6c-4be2-bbf6-a554c2b4eb56 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:14.158453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:14.201014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:14.201918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:14.202314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:14.202698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:14.204595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:14.329064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:14.337918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41cb50-a97a-40d7-b2e8-3f2ac4d6ac98 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:14.360640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 749/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:18:16.821781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:16.822636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:16.822636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:16.824174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 733/1482] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:18:16.833427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c34ec-3594-4cc7-a24a-bca7fd7b8d35 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:16.834287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c34ec-3594-4cc7-a24a-bca7fd7b8d35 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:16.834465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c34ec-3594-4cc7-a24a-bca7fd7b8d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:16.940789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-620c34ec-3594-4cc7-a24a-bca7fd7b8d35 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:16.942973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-620c34ec-3594-4cc7-a24a-bca7fd7b8d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:17.040084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:17.125414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 750/1483] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:17.169599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:17.172552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:17.224289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:17.363132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1da9-d200-4179-a6cc-d77c2933d830 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:17.501995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:17.503135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:17.503631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:17.504152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:17.507479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:17.629209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:17.680010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:17.681628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e014758-5bf4-4eb1-997b-c2a67d6315a5 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:17.833235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 734/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:18:20.206392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:20.207061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:20.207061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:20.208644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 751/1485] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:18:20.215429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0388-82a1-4845-ad28-d06bbc2731ac None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:20.216016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0388-82a1-4845-ad28-d06bbc2731ac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:20.216343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0388-82a1-4845-ad28-d06bbc2731ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:20.341841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bdbb0388-82a1-4845-ad28-d06bbc2731ac None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:20.343593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bdbb0388-82a1-4845-ad28-d06bbc2731ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:20.541789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 735/1486] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:20.608003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:20.610882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:20.822752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43479db6-719b-4476-ac2a-48d26d9ee432 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:20.952506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:20.955355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:20.956943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:20.957601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:20.958120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:20.961905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:21.160736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:21.162299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3d87f-549a-406c-b653-7fa3f9c74c7f service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:21.184251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 752/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:18:23.706357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:23.707295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:23.707295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:23.708720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 736/1488] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:18:23.715077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13cfa-72c9-4f51-a34b-166cfb2e13d0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:23.715969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13cfa-72c9-4f51-a34b-166cfb2e13d0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:23.715969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d13cfa-72c9-4f51-a34b-166cfb2e13d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:23.797013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-04d13cfa-72c9-4f51-a34b-166cfb2e13d0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:23.798312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04d13cfa-72c9-4f51-a34b-166cfb2e13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:23.867051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:23.927257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 753/1489] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:18:23.968887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:23.970706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:24.022578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:24.129991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737954b4-a00a-4019-9b8d-c5b3cee5f088 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:24.229058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:24.229699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:24.230019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:24.230348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:24.232253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:24.331063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:24.382620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:24.384285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf56d9-b4c1-415a-9ac5-7ae1e044b04d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:24.407252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 737/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:18:27.654532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:27.655212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:27.655384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:27.657604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 754/1491] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 25 16:18:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:18:27.666180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284299-d555-4f76-985f-09fad4971325 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:27.667045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284299-d555-4f76-985f-09fad4971325 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:27.667045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284299-d555-4f76-985f-09fad4971325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:27.767615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f5284299-d555-4f76-985f-09fad4971325 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:18:27.768944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f5284299-d555-4f76-985f-09fad4971325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:27.919295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 738/1492] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 25 16:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:18:27.963428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:18:27.965216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:28.131525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6192f-b000-4d8a-a539-740d6befb987 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:18:28.231406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:18:28.232180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:18:28.232593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:18:28.232886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:18:28.234804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:18:28.247503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:28.374163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:18:28.375874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd24c26-efe4-498b-83df-169a743ac382 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:18:28.398361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 755/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:18:37.916896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=82 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:18:38.395417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=50 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:20:18.913698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:18.914685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:18.914685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.076469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:20:19.078291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:19.243066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 739/1494] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:20:19.255047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.255741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.256077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.292355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:20:19.294728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:19.308081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 756/1495] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:20:19.313907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.327477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79156a-8150-44cb-b323-8b99fe94c64f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.334568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.335513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.335902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.336328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.337497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:20:19.337936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.339260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.339260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:20:19.339260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:20:19.340582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.358278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 740/1496] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:20:19.368753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.389193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08768e0b-aad6-410d-9347-4e8aeca3e602 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.398718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.400062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.400526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.401127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.402489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.406426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.406426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:20:19.406426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:20:19.408917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.487649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 757/1497] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:20:19 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:20:19.496674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.516626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652a94-fc62-4df2-9abb-630b8dea8019 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.628340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:20:19.643574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.647237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.647901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.648516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.650569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.652788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.652788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:20:19.652788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:20:19.655569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.703118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 741/1498] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:20:19 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:20:19.712042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.733321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d436-d20c-4686-b2b4-d6817467efaa admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.742841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.744533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.745074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.745653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.747312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.749299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.749299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:20:19.749299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:20:19.751180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.779938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 758/1499] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:20:19.789804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.808231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d97f-7433-4c26-b699-37b8eda4bf8f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.818445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.819847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.820465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.821110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.823205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.825623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.825623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:20:19.825623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:20:19.827635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7f781-f0da-4922-9f9f-8808a7d9e81f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.861282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 742/1500] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:20:19.868584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:19.890128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4445ec-947e-4e0c-8e8f-4dcc244fbfa3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:19.899373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:19.901098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.901711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.902284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] PATH_INFO: `/v3/projects/b3720625f4ea4f87ac03df89a12156cf/users/6eeaefdf801640809393241b17fdc6b9/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:19.939205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3720625f4ea4f87ac03df89a12156cf, user_id=6eeaefdf801640809393241b17fdc6b9, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:19.941687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:19.941687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:20:19.941687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:20:19.946820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32d1b-8aec-48a0-8c7e-378c3a3be408 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:19.974103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 759/1501] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:20:19 2022] PUT /identity/v3/projects/b3720625f4ea4f87ac03df89a12156cf/users/6eeaefdf801640809393241b17fdc6b9/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:20:19.998770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2ea-cf47-4735-915c-af4f4e7f68a5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:19.999659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2ea-cf47-4735-915c-af4f4e7f68a5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:19.999813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2ea-cf47-4735-915c-af4f4e7f68a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.062875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2b63c2ea-cf47-4735-915c-af4f4e7f68a5 None None] MFA Rules not processed for user `6eeaefdf801640809393241b17fdc6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:20:20.065480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b63c2ea-cf47-4735-915c-af4f4e7f68a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:20.206372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 743/1502] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:20:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:20:20.230559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34f5d-1df4-4f14-b72a-33d1db14d7b8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.231096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34f5d-1df4-4f14-b72a-33d1db14d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.231512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34f5d-1df4-4f14-b72a-33d1db14d7b8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.233552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 760/1503] 10.222.0.66 () {54 vars in 964 bytes} [Mon Jul 25 16:20:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:20:20.241452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114614-dab8-4afe-bcb8-886db21e0fac None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.241982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114614-dab8-4afe-bcb8-886db21e0fac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.242359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114614-dab8-4afe-bcb8-886db21e0fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.325772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e2114614-dab8-4afe-bcb8-886db21e0fac None None] MFA Rules not processed for user `468fc86f361a4c7c91cacc1d90086176`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:20:20.327457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e2114614-dab8-4afe-bcb8-886db21e0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:20.453149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 744/1504] 10.222.0.66 () {58 vars in 1061 bytes} [Mon Jul 25 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:20:20.464863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34f5d-1df4-4f14-b72a-33d1db14d7b8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:20.479217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34f5d-1df4-4f14-b72a-33d1db14d7b8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:20.484364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:20.485197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.485610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.485956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.499143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:20.501392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:20.520879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 761/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:20:20.590369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcfe51-a4b0-4acc-9d9e-cb9574909cdf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.590888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcfe51-a4b0-4acc-9d9e-cb9574909cdf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.591261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcfe51-a4b0-4acc-9d9e-cb9574909cdf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.593170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 745/1506] 10.222.0.66 () {52 vars in 830 bytes} [Mon Jul 25 16:20:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:20:20.598590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:20.731531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67735d03-9bb3-4752-88e2-3e8e98b9cb85 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:20.741040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:20.742748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.743243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.743771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] PATH_INFO: `/v3/users/6eeaefdf801640809393241b17fdc6b9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:20.746781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eeaefdf801640809393241b17fdc6b9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:20.748921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:20.748921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:20:20.748921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:20:20.751560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:20.858792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eeaefdf801640809393241b17fdc6b9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:20.890795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eeaefdf801640809393241b17fdc6b9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:20.892585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eeaefdf801640809393241b17fdc6b9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:20.913912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eeaefdf801640809393241b17fdc6b9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:20.915414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eeaefdf801640809393241b17fdc6b9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:20.957456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:20:20.983030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 762/1507] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:20:20 2022] DELETE /identity/v3/users/6eeaefdf801640809393241b17fdc6b9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:20:20.992096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:20.992656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:20.996412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:21.039080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:20:21.041964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:21.063898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:20:21.178575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 746/1508] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:20:21.196127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:21.198864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:21.377753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde8f2-cda2-4c81-b5c1-0779ae696a9b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:21.388917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:21.388917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:21.388917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:21.388917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:21.407666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:21.410813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8bd5c3-8339-40a4-aea6-242bf0ff99ca service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:21.431003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 763/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:20:22.151533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:20:22.153702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:20:22.174834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074d510-4463-40c6-9f93-ec9910fe6e95 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:20:22.186349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:20:22.188233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:20:22.188705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:20:22.189101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] PATH_INFO: `/v3/projects/b3720625f4ea4f87ac03df89a12156cf` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:20:22.191812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3720625f4ea4f87ac03df89a12156cf)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:20:22.194000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:20:22.194000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:20:22.194000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:20:22.197015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:20:22.325472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3720625f4ea4f87ac03df89a12156cf', 'request_id': 'req-3d636abc-54e1-4371-b57c-5f406ce1bb83'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:22.345755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3720625f4ea4f87ac03df89a12156cf', 'request_id': 'req-3d636abc-54e1-4371-b57c-5f406ce1bb83'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:22.359353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3720625f4ea4f87ac03df89a12156cf', 'request_id': 'req-3d636abc-54e1-4371-b57c-5f406ce1bb83'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:22.398740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] The token cache is being invalidate because project b3720625f4ea4f87ac03df89a12156cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:20:22.399119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:22.400566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3d636abc-54e1-4371-b57c-5f406ce1bb83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:20:22.403401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 747/1510] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:20:22 2022] DELETE /identity/v3/projects/b3720625f4ea4f87ac03df89a12156cf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:20:31.422016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:20:32.392284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:22:09.351430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:09.352297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:09.352297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:09.353693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:22:09.359679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fd75-b5da-45e6-a1af-5611374be65f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:09.360565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fd75-b5da-45e6-a1af-5611374be65f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:09.360565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fd75-b5da-45e6-a1af-5611374be65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:09.384694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f637fd75-b5da-45e6-a1af-5611374be65f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:09.386099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f637fd75-b5da-45e6-a1af-5611374be65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:09.466372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:22:09.631915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:09.633777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:09.802959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25cf3d-040b-4d33-9323-356298a2db55 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:09.809227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:09.809958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:09.810244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:09.810527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:09.822169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:09.824036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1776b-d5e4-4410-a2bc-0d770bdf4924 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:09.836223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 765/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:10.726666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838a6d3-548a-4ab6-99c7-5706cc68d79e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:10.727292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838a6d3-548a-4ab6-99c7-5706cc68d79e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:10.727292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838a6d3-548a-4ab6-99c7-5706cc68d79e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:10.729260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:22:10.736998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364a8-d4e8-4a69-9352-59bf88c3e4e8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:10.737656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364a8-d4e8-4a69-9352-59bf88c3e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:10.738011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364a8-d4e8-4a69-9352-59bf88c3e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:10.767040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5f2364a8-d4e8-4a69-9352-59bf88c3e4e8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:10.768385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5f2364a8-d4e8-4a69-9352-59bf88c3e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:10.782883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:10.941124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838a6d3-548a-4ab6-99c7-5706cc68d79e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:10.957801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838a6d3-548a-4ab6-99c7-5706cc68d79e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:10.964417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:10.965119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:10.965434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:10.965717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:10.976733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:10.978435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81106dc3-6b4e-46a2-aceb-e68611c5217e service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:10.989733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 750/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:11.186233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e3565-d352-4ebc-9f06-ec1b09457c75 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:11.186567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e3565-d352-4ebc-9f06-ec1b09457c75 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:11.186868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e3565-d352-4ebc-9f06-ec1b09457c75 None None] PATH_INFO: `/v3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:11.188420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 767/1517] 10.222.0.102 () {54 vars in 926 bytes} [Mon Jul 25 16:22:11 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:22:11.195474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669856e2-9e24-4156-908e-6ce422877bc0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:11.195815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669856e2-9e24-4156-908e-6ce422877bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:11.196209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669856e2-9e24-4156-908e-6ce422877bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:11.271440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-669856e2-9e24-4156-908e-6ce422877bc0 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:11.272727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-669856e2-9e24-4156-908e-6ce422877bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:11.372154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 751/1518] 10.222.0.102 () {58 vars in 1016 bytes} [Mon Jul 25 16:22:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:15.233971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bd1aa-8791-4969-9fef-e7a7bcf179a1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:15.234559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bd1aa-8791-4969-9fef-e7a7bcf179a1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:15.234747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bd1aa-8791-4969-9fef-e7a7bcf179a1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:15.236272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 768/1519] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 25 16:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:15.243495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821cb-8f0f-42b8-adf4-e54c42168e41 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:15.244112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821cb-8f0f-42b8-adf4-e54c42168e41 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:15.244112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821cb-8f0f-42b8-adf4-e54c42168e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:15.273935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-238821cb-8f0f-42b8-adf4-e54c42168e41 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:15.276015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-238821cb-8f0f-42b8-adf4-e54c42168e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:15.291482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 752/1520] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 25 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:15.305691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bd1aa-8791-4969-9fef-e7a7bcf179a1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:15.329280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bd1aa-8791-4969-9fef-e7a7bcf179a1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:15.338581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:15.339776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:15.340272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:15.340780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:15.357722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:15.360755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822f37-18fb-4064-a365-cca9a3169f7f service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:15.382189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 769/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:22.276947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0535a1-b052-4b1c-af33-c0c41dc1e67f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:22.277893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0535a1-b052-4b1c-af33-c0c41dc1e67f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:22.277893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0535a1-b052-4b1c-af33-c0c41dc1e67f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:22.278969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:22:22.286029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444447-acbb-4586-83fa-6719ac383f49 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:22.286611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444447-acbb-4586-83fa-6719ac383f49 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:22.286889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444447-acbb-4586-83fa-6719ac383f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:22.315664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-92444447-acbb-4586-83fa-6719ac383f49 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:22.317077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-92444447-acbb-4586-83fa-6719ac383f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:22.325800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:22.489207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0535a1-b052-4b1c-af33-c0c41dc1e67f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:22.506243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0535a1-b052-4b1c-af33-c0c41dc1e67f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:22.515817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:22.516501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:22.516780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:22.517055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:22.527211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:22.528910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f55c3-8fae-4b4d-b800-d2b3cf277a50 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:22.541497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 754/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:23.436132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceec03-b701-4388-ac63-d94336ccee5b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:23.436938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceec03-b701-4388-ac63-d94336ccee5b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:23.436938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceec03-b701-4388-ac63-d94336ccee5b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:23.438317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 16:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:23.444375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8fe5-aba9-4e1b-83e6-45d125487558 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:23.444827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8fe5-aba9-4e1b-83e6-45d125487558 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:23.444929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8fe5-aba9-4e1b-83e6-45d125487558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:23.466805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-603f8fe5-aba9-4e1b-83e6-45d125487558 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:23.468125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-603f8fe5-aba9-4e1b-83e6-45d125487558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:23.477142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 16:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:22:23.635096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceec03-b701-4388-ac63-d94336ccee5b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:23.652233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceec03-b701-4388-ac63-d94336ccee5b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:23.659060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:23.659773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:23.660080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:23.660472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:23.670874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:23.672940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5050aa5-369b-4e95-aaa8-8727439e44bc service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:23.685040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 772/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:22:27.793614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:27.794573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:27.794573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:27.795674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:27.803124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b7d2-8c6e-4d74-8086-8e99cffa5e80 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:27.803743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b7d2-8c6e-4d74-8086-8e99cffa5e80 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:27.804140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b7d2-8c6e-4d74-8086-8e99cffa5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:27.829237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6561b7d2-8c6e-4d74-8086-8e99cffa5e80 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:27.830558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6561b7d2-8c6e-4d74-8086-8e99cffa5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:27.839677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:27.965761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:27.967906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:28.008549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:22:28.047801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bcc65-cde8-4521-926b-91159abbe6bb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:28.051971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:28.052658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:28.052945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:28.053227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:28.064428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:28.066047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6459db-16a7-4802-b693-7941ee87bfb1 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:28.078215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 757/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:30.380398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6098bb-bcc5-4e17-861e-5be483079f30 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:30.381070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6098bb-bcc5-4e17-861e-5be483079f30 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:30.381070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6098bb-bcc5-4e17-861e-5be483079f30 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:30.382504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:30.389684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afff28-a5e2-4d2b-aced-fb2de5e744af None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:30.390600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afff28-a5e2-4d2b-aced-fb2de5e744af None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:30.390600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afff28-a5e2-4d2b-aced-fb2de5e744af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:30.412974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-08afff28-a5e2-4d2b-aced-fb2de5e744af None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:30.414254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-08afff28-a5e2-4d2b-aced-fb2de5e744af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:30.424584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:30.464673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6098bb-bcc5-4e17-861e-5be483079f30 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:30.483388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6098bb-bcc5-4e17-861e-5be483079f30 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:30.488554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:30.489316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:30.489800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:30.490206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:30.502606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:30.504260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb6459-9164-4039-9919-6dff2d985c1d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:30.516064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 775/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:34.541554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:34.542221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:34.542221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:34.543723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:34.550122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706dcff-9dce-4177-b0e3-8945f459434b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:34.551034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706dcff-9dce-4177-b0e3-8945f459434b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:34.551034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706dcff-9dce-4177-b0e3-8945f459434b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:34.562768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=74 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:22:34.584161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0706dcff-9dce-4177-b0e3-8945f459434b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:34.586568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0706dcff-9dce-4177-b0e3-8945f459434b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:34.601470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:22:34.941420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:34.943442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:35.024246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c1b2c-a14b-4de2-9048-b48d23db10bc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:35.028386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:35.029079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:35.029473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:35.029747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:35.040732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:35.042285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11317d48-2124-4088-8c68-c92c72564a7b service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:35.053657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 760/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:37.264049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:37.264894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:37.264894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:37.266143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 16:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:22:37.272433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca665bd-44a7-48c3-b582-ef037a9fa562 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:37.273103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca665bd-44a7-48c3-b582-ef037a9fa562 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:37.273103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca665bd-44a7-48c3-b582-ef037a9fa562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:37.296642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7ca665bd-44a7-48c3-b582-ef037a9fa562 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:22:37.297945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7ca665bd-44a7-48c3-b582-ef037a9fa562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:37.307107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 16:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:22:37.630261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:22:37.632166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:22:37.646945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a394391-7614-4f46-8a9a-b6233446b18e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:22:37.651486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:22:37.652241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:22:37.652652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:22:37.653092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:22:37.664937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:22:37.666599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf00a0-088a-4ab3-b61f-5d91d4e8eee2 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:22:37.678312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 778/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:22:47.290017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=47 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:22:47.673135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=8 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:15.199769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.201158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.201708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.231666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:15.233709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:15.251381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 762/1540] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:24:15.272743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.273923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.273923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.328506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:15.330724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:15.351725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 779/1541] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:15.357831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.377290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cce0c-a036-4c2e-82f0-6816f4e63b0d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.388393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.389764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.391005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.391558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.392583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:15.393224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:15.395258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:15.416227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 763/1542] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:15.426683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.449695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceade8ed-3c51-4abb-bcca-e231f04c4431 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.465116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.466167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.466641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.467114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.468413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:15.472405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:15.553419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 780/1543] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:15 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:15.561227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.592623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b25d4-dc89-4c05-80b4-64c0d2541823 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.714610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.715901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.716436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.716933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.717950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:15.720732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:15.767772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 764/1544] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:15 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:15.774161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.807048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d5dff-7245-4235-9323-34cd325b64bc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.825064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.826337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.826838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.827340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.828861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:15.832196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:15.867710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 781/1545] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:15.872544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.893423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ba90-fe7d-404e-bc49-ce927cf0a4bc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.910888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.912288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.913118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.913836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:15.914973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:15.917539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6a567-3be6-4b8b-b815-6914fc954909 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:15.936081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 765/1546] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:15.942098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:15.968376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b30-e240-4871-b71f-9273af53ea77 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:15.984601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:15.986510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:15.987029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:15.987720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] PATH_INFO: `/v3/projects/287b6d7980174314b20fe6b172725bf4/users/f2d0f7e3f1cc4955a0ca99d588344322/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.027660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=287b6d7980174314b20fe6b172725bf4, user_id=f2d0f7e3f1cc4955a0ca99d588344322, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:16.036457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:16.073399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 782/1547] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:15 2022] PUT /identity/v3/projects/287b6d7980174314b20fe6b172725bf4/users/f2d0f7e3f1cc4955a0ca99d588344322/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:16.080021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621b10e-294c-4044-af5e-f3b177bf6284 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.080385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621b10e-294c-4044-af5e-f3b177bf6284 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.080622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621b10e-294c-4044-af5e-f3b177bf6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.122821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e621b10e-294c-4044-af5e-f3b177bf6284 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:16.124099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e621b10e-294c-4044-af5e-f3b177bf6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:16.231256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 766/1548] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:24:16.262292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:16.344624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9340-9bf3-49a6-ad35-b058092ca743 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.345017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9340-9bf3-49a6-ad35-b058092ca743 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.345209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9340-9bf3-49a6-ad35-b058092ca743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.411253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6eaf9340-9bf3-49a6-ad35-b058092ca743 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:16.412916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6eaf9340-9bf3-49a6-ad35-b058092ca743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:16.449076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 767/1549] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:16.450433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a6ec-5fe4-42f1-a11e-b72ee6bcaed6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:16.458515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.458958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.459279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.465490 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:16.466182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.466466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.466744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.513219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:16.514961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:16.522718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:16.523999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:16.543484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 783/1550] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:16.543745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 768/1551] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:24:16.553890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:16.590160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03699d6-3f4c-4104-968b-ee9828e6f8fb service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:16.607082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:16.607082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.607082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.607082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.607969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:16.611386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:16.611386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:16.611386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:24:16.611386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:24:16.615440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:16.647647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 784/1552] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:16.670808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:16.692892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3cd47-b0f6-4713-ad74-8d10db8e6c05 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:16.707243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:16.708525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:16.708964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:16.709402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:16.710621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:16.718770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:16.718770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:24:16.718770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:24:16.720245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:16.795547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 769/1553] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:16 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:16.802393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:16.828357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785fec-c092-4274-abf0-0aba09bc756e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:17.019619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:17.020186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.020627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.021125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.022471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:17.024396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:17.024396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:24:17.024396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:24:17.036172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:17.121263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 785/1554] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:16 2022] POST /identity/v3/users => generated 284 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:17.131713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:17.156776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36698d-00b8-4169-ba46-f2fe5494aee8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:17.175413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:17.176197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.176529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.176990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.182136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:17.184723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:17.217521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 770/1555] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:17.219609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:17.257533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961d26-612a-4dc7-92cc-15f721a82311 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:17.274589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:17.276013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.276574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.277090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.278098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:17.285385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c256e-5975-4c1c-8e7e-34ae3041b5ec admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:17.311225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 786/1556] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:17.322601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:17.361820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a66aae-e28a-401f-be70-e375a8dffa2b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:17.384113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:17.387499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.388267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.393404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] PATH_INFO: `/v3/projects/13beb0177fdd4ba9b43a260045fce37e/users/1b025bd914a34faf900b460eaedc1493/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.461391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:17.478328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=13beb0177fdd4ba9b43a260045fce37e, user_id=1b025bd914a34faf900b460eaedc1493, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:17.485388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:17.485388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:24:17.485388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:24:17.495907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520bca1-f6ea-4bb0-8d29-07f1ce583129 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.497720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520bca1-f6ea-4bb0-8d29-07f1ce583129 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.498396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520bca1-f6ea-4bb0-8d29-07f1ce583129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.513601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a13776-f75a-4acc-8003-5a9801a393ee admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:17.553377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:17.589887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 771/1557] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:17 2022] PUT /identity/v3/projects/13beb0177fdd4ba9b43a260045fce37e/users/1b025bd914a34faf900b460eaedc1493/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:17.602512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29ab68-22d1-4997-aa2e-9af4f7619286 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.614380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29ab68-22d1-4997-aa2e-9af4f7619286 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.615209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29ab68-22d1-4997-aa2e-9af4f7619286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.622460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a520bca1-f6ea-4bb0-8d29-07f1ce583129 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:17.630587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a520bca1-f6ea-4bb0-8d29-07f1ce583129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:17.713238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6f29ab68-22d1-4997-aa2e-9af4f7619286 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:17.715918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6f29ab68-22d1-4997-aa2e-9af4f7619286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:17.872882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 787/1558] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:17.877426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8d43b-213f-4122-bfe9-0b450eb685f5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.877821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8d43b-213f-4122-bfe9-0b450eb685f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.878111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8d43b-213f-4122-bfe9-0b450eb685f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.909691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 772/1559] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:17.920429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113efc93-0cc5-4586-b855-45b5662808f5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.920895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113efc93-0cc5-4586-b855-45b5662808f5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.921414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113efc93-0cc5-4586-b855-45b5662808f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:17.927266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f9a8d43b-213f-4122-bfe9-0b450eb685f5 None None] MFA Rules not processed for user `1b025bd914a34faf900b460eaedc1493`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:17.928700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f9a8d43b-213f-4122-bfe9-0b450eb685f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:17.963045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-113efc93-0cc5-4586-b855-45b5662808f5 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:17.965207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-113efc93-0cc5-4586-b855-45b5662808f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:17.979905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 773/1560] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:17.982357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd458d-2ef6-4e13-876a-bf6f529e3634 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:17.982864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd458d-2ef6-4e13-876a-bf6f529e3634 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:17.983432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd458d-2ef6-4e13-876a-bf6f529e3634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.012263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-25cd458d-2ef6-4e13-876a-bf6f529e3634 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.013684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-25cd458d-2ef6-4e13-876a-bf6f529e3634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.033877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 774/1561] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.035726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb58e-49b3-4b08-9dff-3e6593bbe29e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.036299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb58e-49b3-4b08-9dff-3e6593bbe29e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.036783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb58e-49b3-4b08-9dff-3e6593bbe29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.059790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 788/1562] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.061288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba8dcb-e288-4bc2-82b8-f7f08aef9553 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.061635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba8dcb-e288-4bc2-82b8-f7f08aef9553 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.061889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba8dcb-e288-4bc2-82b8-f7f08aef9553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.067020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1b8bb58e-49b3-4b08-9dff-3e6593bbe29e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.068481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1b8bb58e-49b3-4b08-9dff-3e6593bbe29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.079718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 775/1563] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.081269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3823-013e-49ea-a4ed-2582af63d10d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.081606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3823-013e-49ea-a4ed-2582af63d10d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.081864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3823-013e-49ea-a4ed-2582af63d10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.101967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cbba8dcb-e288-4bc2-82b8-f7f08aef9553 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.105089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cbba8dcb-e288-4bc2-82b8-f7f08aef9553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.112187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-42cd3823-013e-49ea-a4ed-2582af63d10d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.113526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-42cd3823-013e-49ea-a4ed-2582af63d10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.119968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 789/1564] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.126833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 776/1565] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.129465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826dc8d-1e7d-416d-88c4-6974d391fb62 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.129888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826dc8d-1e7d-416d-88c4-6974d391fb62 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.130218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826dc8d-1e7d-416d-88c4-6974d391fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.134476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.134973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.135575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.160575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3826dc8d-1e7d-416d-88c4-6974d391fb62 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.162839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3826dc8d-1e7d-416d-88c4-6974d391fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.172824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.173555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 777/1566] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.175990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.183529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.183819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.184198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.193154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 790/1567] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.195115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.223575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.226601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.253699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 778/1568] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.255448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.272189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faf89b-178e-4639-ade5-925ee5af664e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.283622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.284792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.285415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.285973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.287133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.287576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.289612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.308257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 779/1569] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.309377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.321710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703dd8-e465-4f34-b194-1bbc266f1297 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.324530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44922437-1215-40ee-b3a3-71aa695929e5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.327916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.328657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.328950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.329274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.336170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.339828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.342074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003734-e1c6-4e43-8d65-59282520dcb0 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.349954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.350823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.351411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.352493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.353001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.353310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 791/1570] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:18.354963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.355332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.355605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.361204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea52c8-bfef-486b-a838-d4436872929e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.390782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.392623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.394257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 780/1571] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.396807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.397299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.397830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.407605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 792/1572] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.408751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.434239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4175ca-fb48-41e7-9fe9-34998866ec30 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.443383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:18.444467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.446336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:18.448160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.449017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.449864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.452042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.453128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.458634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.462905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 781/1573] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:18.463214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.496598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fddaf-5a4c-4629-b8a3-5e073bbc42cd None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.518488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 793/1574] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.518488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.519417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.519417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.519417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.519417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.519843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.519925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.526610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:18.532361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9bfab-4ca2-4609-b32e-62f46ed1fad1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.537450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.541123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.542266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.543134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.544643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.546173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.549244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.588756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 782/1575] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.591875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 794/1576] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:18.593295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.597187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.639475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634477-a774-4eb6-a8a5-acb626c812e7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.704818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a46b9-174b-474c-a64e-285505742150 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.711836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.713911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.714845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.715336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.716583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.717007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.719962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.739409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 795/1577] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.740549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.767834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998a372-f90f-420e-965d-dbb13c4bd714 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.776718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.778572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.779103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.779671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.781039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:18.781673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.784983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.810489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 796/1578] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:18.811793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.828410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.829227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.829670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.830069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.830828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.833538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.838230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafea4f-e092-4cbe-b844-9b6ccc1d75b9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:18.849593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:18.857845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:18.858958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:18.859602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:18.860116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:18.860821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:18.863118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:18.899853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 783/1579] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:18.901707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:18.914211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 797/1580] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:18.915910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.048486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b346-0c56-4459-984e-1b1c694fde9a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.053655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.054440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.054725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.055004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.056198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.058499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.088241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 798/1581] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.089570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.098294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993afe7-75fa-4832-9615-d45599c9551a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.103354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec275-c360-4018-99dc-1f5fbab89021 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.106958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.108106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.108600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.109089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.110257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.111020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.111306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.111582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.112408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.115497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.122070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.126875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.158818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 799/1582] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.159632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.175980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:19.264617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 784/1583] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:18 2022] POST /identity/v3/projects => generated 385 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.264617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.348503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb71ef-60d3-4dfc-9cc5-535a56521c22 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.528322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57895ca-7ab1-4b8f-8b45-5133b79ab3f7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.542984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.547224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.550029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.551222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.553323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.558585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.559482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.560887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.561655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.562293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.563934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.575034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.624378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 785/1584] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/users => generated 287 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.626048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.653772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cca720-c1df-49c6-80f7-25427e7bd8c7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.667070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.668812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.669019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 800/1585] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/projects => generated 393 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.669918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.672709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.673445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.676262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.682105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.725507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 786/1586] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:19.727040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.752545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e696-3acf-4ac1-b6c8-dbe4c79b75ac admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.896143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e068ac-43bc-49de-a935-f811e00d3ec3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.909480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:19.913168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.915146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.922004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.922537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.923881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.928200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:19.952689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 801/1587] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:19.954179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:19.971063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601188b-38e0-477e-ae4e-f04e52d2a404 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:19.984942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:19.985751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:19.986378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:19.986696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:19.987468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:19.991269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.005969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.007226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.007726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.008210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.009824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.014093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.043094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 802/1588] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:20.045206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.067908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a760b-1f51-4f42-9a88-f414b0d2dc29 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.074098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 787/1589] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:19 2022] POST /identity/v3/users => generated 288 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:20.075550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.081718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.083425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.083911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.084413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.085867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.088151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.096120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf32f-a2a0-424c-85f8-235b306e2cd0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.108509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.109845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.110566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.111002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.112742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.114698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 803/1590] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.115749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.116599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.135196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949549b5-0ac8-468f-abed-d45f16568ca7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.143592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.145802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.146273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.146606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 788/1591] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:20 2022] POST /identity/v3/users => generated 297 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:20.147206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.148607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.151213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.151401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.155273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:20.166086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd6782-7722-4cfa-901a-e569db1ab0d7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.172210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.172451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 804/1592] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.172787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.173040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.173317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.173683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.174135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.175845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.195857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a704-e848-4533-81ad-3a94d064fd37 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.204504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 789/1593] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.204504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.206273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.207522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.208111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.209050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.210782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.214032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.223589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca358e87-d51a-4dcc-a2f9-4672816a6c9f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.233733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.235366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.235526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 805/1594] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.235616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.236123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.237280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.237576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.241717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.255679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f03c93-b9ff-4131-807b-1a956c53daa5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.267933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 790/1595] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.268278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.268462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.268595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.268595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.268595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.268595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.270144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.296320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 806/1596] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.296320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.297814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a904-f5f1-4cbc-b68b-64ffaaa96d01 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.308710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.310169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.310700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.311270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.312941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.316493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299bd7-b169-4c51-89a5-4cc9aed4d528 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.316790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.330377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.331230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.331583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.331928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.333910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.337961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.342413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 791/1597] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.344279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.351715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 807/1598] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.353551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.364938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be4aff-19ab-4bc7-9182-70aab5e968d2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.373307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732812d4-6d0f-4eb1-9176-a7b03a68e086 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.379327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.381301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.381935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.382431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] PATH_INFO: `/v3/projects/157fa0d8c8d04d099ce05b715300e22b/users/716072978f3948419a06e14ef4746de8/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.384551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.386135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.386757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.387248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.390049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.397723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.431371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 808/1599] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.431613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157fa0d8c8d04d099ce05b715300e22b, user_id=716072978f3948419a06e14ef4746de8, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.433769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.441188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.461785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658735-bef3-4189-b1e8-f9c7b0b5dff5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.473906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 792/1600] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:20 2022] PUT /identity/v3/projects/157fa0d8c8d04d099ce05b715300e22b/users/716072978f3948419a06e14ef4746de8/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:20.485958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.641496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.643508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.644049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.644437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.647817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.649271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.676071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 809/1601] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.676071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.699010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcdd2b-e8f9-4b0b-87c4-2f04481b0dd1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.707802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.708563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.708991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.709529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.710947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.714353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.719935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b516-4211-453d-8023-41d3e7a12299 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.738211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 810/1602] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:20.739232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.739470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.740282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.740651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.740820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] PATH_INFO: `/v3/projects/f57d34a39f87447f82b90b67a29cf2c8/users/3a30c39e01734a7c8630e9ed28cbf510/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.764843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f583f-13bf-421f-892d-c0fec37cb3e8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:20.776708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:20.778487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.778996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.779501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] PATH_INFO: `/v3/projects/3e24e8cd4aa84e75b511d18e6ac7c197/users/b0bfbe80c5b84bbdac6af9e6595a8bc9/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.796605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57d34a39f87447f82b90b67a29cf2c8, user_id=3a30c39e01734a7c8630e9ed28cbf510, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.800684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.834877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 793/1603] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:20 2022] PUT /identity/v3/projects/f57d34a39f87447f82b90b67a29cf2c8/users/3a30c39e01734a7c8630e9ed28cbf510/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:20.834877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:20.848076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e24e8cd4aa84e75b511d18e6ac7c197, user_id=b0bfbe80c5b84bbdac6af9e6595a8bc9, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:20.853788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd51a21-f40a-43cb-8561-eda2feca5b33 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:20.899784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 811/1604] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:20 2022] PUT /identity/v3/projects/3e24e8cd4aa84e75b511d18e6ac7c197/users/b0bfbe80c5b84bbdac6af9e6595a8bc9/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:20.916089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:20.916624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:20.917148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:20.928701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:20.961327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7755-7ded-49c2-9e91-9c3968dcc610 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:21.024733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] MFA Rules not processed for user `716072978f3948419a06e14ef4746de8`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.028545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.116251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:21.117551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.117928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.118446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] PATH_INFO: `/v3/projects/0ab28729cacc4da1a5025f52ac5f1127/users/ed49dd65d33b408a8d4850c30e4e8a25/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.174368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab28729cacc4da1a5025f52ac5f1127, user_id=ed49dd65d33b408a8d4850c30e4e8a25, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:21.179047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:21.206366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 794/1605] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:20 2022] PUT /identity/v3/projects/0ab28729cacc4da1a5025f52ac5f1127/users/ed49dd65d33b408a8d4850c30e4e8a25/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:21.220220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:21.238860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 812/1606] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.238860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:21.253479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:21.263400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ab59a-e112-448c-b1db-38167c867801 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:21.356781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:21.358553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.359118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.359621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] PATH_INFO: `/v3/projects/18b6e0eaf4eb48868a683c7769d3df7a/users/2d1eb89923784f1c83d2077c6b6926e9/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.376658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c45a-962c-4e9b-b11a-01982357420f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:21.384146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b6e0eaf4eb48868a683c7769d3df7a, user_id=2d1eb89923784f1c83d2077c6b6926e9, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:21.385694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:21.387371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.387883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.388382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] PATH_INFO: `/v3/projects/5c4be8083168475198c66cca890b18a9/users/644c0a532fa747c7abf4d53759e89181/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.389205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f7d14-4227-401a-9d92-20d10eaf7c13 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:21.414871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 813/1607] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:21 2022] PUT /identity/v3/projects/18b6e0eaf4eb48868a683c7769d3df7a/users/2d1eb89923784f1c83d2077c6b6926e9/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:21.417020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e4f8-70e7-4695-b56d-e4f366a90d12 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.417837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e4f8-70e7-4695-b56d-e4f366a90d12 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.419388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e4f8-70e7-4695-b56d-e4f366a90d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.425191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4be8083168475198c66cca890b18a9, user_id=644c0a532fa747c7abf4d53759e89181, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:21.430969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7be140-7827-4611-9579-e345c41f799b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:21.457302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 795/1608] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:21 2022] PUT /identity/v3/projects/5c4be8083168475198c66cca890b18a9/users/644c0a532fa747c7abf4d53759e89181/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:21.459529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.460043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.460544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.476828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9a09e4f8-70e7-4695-b56d-e4f366a90d12 None None] MFA Rules not processed for user `3a30c39e01734a7c8630e9ed28cbf510`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.478631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9a09e4f8-70e7-4695-b56d-e4f366a90d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.513120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] MFA Rules not processed for user `b0bfbe80c5b84bbdac6af9e6595a8bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.514889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.606100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:21.606854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 814/1609] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.609649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c73f-accb-4533-8870-b26fa884c1d0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.610160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c73f-accb-4533-8870-b26fa884c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.610631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c73f-accb-4533-8870-b26fa884c1d0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.617193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 815/1610] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 25 16:24:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:21.620208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff774b-9bca-4c3d-9bf4-e4bae7a74038 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.621052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff774b-9bca-4c3d-9bf4-e4bae7a74038 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.621553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff774b-9bca-4c3d-9bf4-e4bae7a74038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.684690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 796/1611] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.692752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:21.699160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d9ff774b-9bca-4c3d-9bf4-e4bae7a74038 None None] MFA Rules not processed for user `ed49dd65d33b408a8d4850c30e4e8a25`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.701793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d9ff774b-9bca-4c3d-9bf4-e4bae7a74038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.807580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 816/1612] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.809704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f1c0-d097-43ef-bc64-48a4fb464bb6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.809996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f1c0-d097-43ef-bc64-48a4fb464bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.810292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f1c0-d097-43ef-bc64-48a4fb464bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.837665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:21.847925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1fd5f1c0-d097-43ef-bc64-48a4fb464bb6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.849720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1fd5f1c0-d097-43ef-bc64-48a4fb464bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.858314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d01ae-4789-47ff-9610-8eec4f3672b7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:21.863510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 817/1613] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.866394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df698-cdae-4bb5-a1fa-af52740892bd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.867152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df698-cdae-4bb5-a1fa-af52740892bd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.867804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df698-cdae-4bb5-a1fa-af52740892bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.868637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:21.870482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.871410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.871981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.873452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:21.877949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d8b40-438c-4e27-b8f2-f3fae9ae7ad9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:21.916050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3b4df698-cdae-4bb5-a1fa-af52740892bd None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:21.917562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3b4df698-cdae-4bb5-a1fa-af52740892bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:21.934005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 818/1614] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:21.934005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6f2d-239e-4307-839d-22181b583131 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.934005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6f2d-239e-4307-839d-22181b583131 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.934005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6f2d-239e-4307-839d-22181b583131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.935351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 797/1615] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:21.937280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb498-cd2a-4dcf-b4a0-db3a8a3a0f61 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:21.937791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb498-cd2a-4dcf-b4a0-db3a8a3a0f61 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:21.938193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb498-cd2a-4dcf-b4a0-db3a8a3a0f61 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:21.941069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 798/1616] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 25 16:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:21.942336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fb498-cd2a-4dcf-b4a0-db3a8a3a0f61 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.050367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-486e6f2d-239e-4307-839d-22181b583131 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:22.050700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fb498-cd2a-4dcf-b4a0-db3a8a3a0f61 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.053398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-486e6f2d-239e-4307-839d-22181b583131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:22.077287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.078844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.081297 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.082073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.082974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.091819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.178085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 799/1617] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:22.179307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.265806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 819/1618] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 25 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:22.266754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e6f2d-239e-4307-839d-22181b583131 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.280198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e6f2d-239e-4307-839d-22181b583131 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.293091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574fd93-0c4c-4463-a363-182cad260e93 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.297799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.298850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.299350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.299931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.311600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:22.391143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.392491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.392951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.393434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.394332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.397154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf0c6f-6480-4123-8ed6-8ea8e43e1436 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.414151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.416486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.431890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 800/1619] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:22.434955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.447773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 820/1620] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:22 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:22.449679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.449987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.450270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.461195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc757cd-2d76-45b5-94ed-891d355026a0 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.481916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.483408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.484153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.484647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.495108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:22.496476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:22.514800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.518097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.541158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 801/1621] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:22.543557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.567128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba355b8f-3b67-4c19-9df7-c14e9cb89a60 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.576596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.577935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.578474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.578966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.580582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.585067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279813e-0344-4aed-891c-89b888049576 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.601435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:22.641934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 802/1622] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:22 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:22.648472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdbbe-8274-4db8-874e-7208caaff36d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.648692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdbbe-8274-4db8-874e-7208caaff36d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.649176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdbbe-8274-4db8-874e-7208caaff36d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.652840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 803/1623] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 25 16:24:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:22.656151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cdbbe-8274-4db8-874e-7208caaff36d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.672771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 821/1624] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 25 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:22.674523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.695779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6466b-9d9c-4a0e-bab5-023042833912 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.704482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.705823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.706322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.706825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.708258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.711802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce76b93-2277-46ec-80a7-410a9e03e65d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.725495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cdbbe-8274-4db8-874e-7208caaff36d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.736403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.736895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 822/1625] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:22.739764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.740279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.740795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.745821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.745821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.745821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.746990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.750235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.775810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 804/1626] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:22.775810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.798370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec3b88-265e-4e31-bddb-56b3662271bf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.807370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.808503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.808961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.809543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.812578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:22.818353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:22.842413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 823/1627] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:22.844194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.876143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6592be-8dd2-4260-8b91-273de0c93bce None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:22.888994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:22.891015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:22.891507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:22.891994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] PATH_INFO: `/v3/projects/ef072b299931436dae8a9635e8dd08cb/users/4ff8ea0aa3924316aad19da48046cf59/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:22.932087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef072b299931436dae8a9635e8dd08cb, user_id=4ff8ea0aa3924316aad19da48046cf59, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.938711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.955654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:22.961643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:22.969612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 824/1628] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:22 2022] PUT /identity/v3/projects/ef072b299931436dae8a9635e8dd08cb/users/4ff8ea0aa3924316aad19da48046cf59/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:22.971453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:22.987168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 805/1629] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:23.002679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.192989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca17bb-e22a-4d92-8d34-b74cbdbb56fa service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.200437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b742a-1275-4d03-9bbe-ce120fe55be8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.217877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.218977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.219442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.219822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] PATH_INFO: `/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38/users/0af1812a4a694689ad9fbf038815a1a6/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.251559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:23.274772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10df4de2a8e94cd6a95adf7e42cd8a38, user_id=0af1812a4a694689ad9fbf038815a1a6, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.280659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:23.310921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 825/1630] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:22 2022] PUT /identity/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38/users/0af1812a4a694689ad9fbf038815a1a6/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:23.315396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.346184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.346901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.347187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.347498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.459305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.461637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:23.493236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 806/1631] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:23.497584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.527144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c3e7-c2df-4f1a-9443-d7a21735c0f7 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.589845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cfbb6-38fa-4dcd-b59c-a5cfe44cd792 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.599210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.600506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.601046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.601559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.603596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.606586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6b9c2-3d7b-40b3-898b-61d295625561 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:23.620947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.622105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.622400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.622680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.623486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.626700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:23.636183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 826/1632] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:23.638802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dabd14-9346-43a5-b67e-0f100cc8cdab None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.639281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dabd14-9346-43a5-b67e-0f100cc8cdab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.640417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dabd14-9346-43a5-b67e-0f100cc8cdab None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.643543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 827/1633] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 25 16:24:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:23.644982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dabd14-9346-43a5-b67e-0f100cc8cdab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.647401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 807/1634] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:23.648524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.664285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a465c0-3f20-43b2-811c-73f934022c75 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.675728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.677225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.677905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.678390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] PATH_INFO: `/v3/projects/ef072b299931436dae8a9635e8dd08cb/users/4ff8ea0aa3924316aad19da48046cf59/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.682393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef072b299931436dae8a9635e8dd08cb, user_id=4ff8ea0aa3924316aad19da48046cf59, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.686998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:23.704692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 808/1635] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:23 2022] PUT /identity/v3/projects/ef072b299931436dae8a9635e8dd08cb/users/4ff8ea0aa3924316aad19da48046cf59/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:23.706206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:23.748235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:23.869866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:23.898037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dabd14-9346-43a5-b67e-0f100cc8cdab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.914831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91727341-13a4-4a73-a1d8-2e0087fe733c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:23.936631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:23.939325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:23.939928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:23.940663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] PATH_INFO: `/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38/users/0af1812a4a694689ad9fbf038815a1a6/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:23.945502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10df4de2a8e94cd6a95adf7e42cd8a38, user_id=0af1812a4a694689ad9fbf038815a1a6, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:23.952498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fea5b-ff33-4434-a0f0-bc444d8204c0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:24.017379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 809/1636] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:23 2022] PUT /identity/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38/users/0af1812a4a694689ad9fbf038815a1a6/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:24.017379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a85d-ec4b-4f77-8ce7-5165a50aaf8d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:24.017379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a85d-ec4b-4f77-8ce7-5165a50aaf8d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:24.018747 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a85d-ec4b-4f77-8ce7-5165a50aaf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:24.059838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:24.061172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:24.061861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:24.062655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:24.085215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6ec8a85d-ec4b-4f77-8ce7-5165a50aaf8d None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:24.090200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6ec8a85d-ec4b-4f77-8ce7-5165a50aaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:24.268733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:24.268733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeaaf0-7e14-4de9-902a-ed80646768a2 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:24.268733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 828/1637] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:24.289323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:24.291751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:24.293747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:24.329147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 810/1638] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:24.333386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:24.336578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:24.336578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:24.465891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] MFA Rules not processed for user `4ff8ea0aa3924316aad19da48046cf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:24.478102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:24.514888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] MFA Rules not processed for user `0af1812a4a694689ad9fbf038815a1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:24.519036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:24.637270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:24.712841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 829/1639] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:24.712841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:24.803830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 811/1640] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:24.892166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648224-6e81-4b3d-b8bd-fae2a660ed23 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:24.910058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:24.913410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:24.914497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:24.915580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:24.951483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:24.957958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:25.013429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 830/1641] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:25.500522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:25.533054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252152cc-0208-49a9-a3a5-86b529966660 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:25.552885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:25.555375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:25.555993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:25.556505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:25.619181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:25.630227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:25.678890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 812/1642] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:25.787405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:25.825311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fd2b-1d51-4ae9-b971-9d44e3c36525 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:25.852467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:25.853750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:25.854152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:25.854629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:25.931171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:25.934180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:26.009533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 831/1643] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:25 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:26.527105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:26.581961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820d89-c03b-4b38-8288-46424a2b79a5 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:26.618630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:26.619813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:26.620341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:26.621226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:26.791776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:26.821855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:26.865653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff491290-84ae-43c4-a85b-7b1d096b6c48 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:26.867466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:26.879864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3ed56-1039-4ad2-b062-dbce3c885be3 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:26.891871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:26.894009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:26.895113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:26.895948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:26.938014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 813/1644] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:27.141668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:27.148023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304466e7-4769-475d-a595-9ec5d04582ef service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:27.194735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 832/1645] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:29.496870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:29.497528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:29.498007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:29.641365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] MFA Rules not processed for user `f2d0f7e3f1cc4955a0ca99d588344322`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:29.642975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:29.830539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 814/1646] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:29.855856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19141249-e687-4535-a957-a114080f27f4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:29.856597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19141249-e687-4535-a957-a114080f27f4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:29.856597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19141249-e687-4535-a957-a114080f27f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:29.896671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-19141249-e687-4535-a957-a114080f27f4 None None] MFA Rules not processed for user `f2d0f7e3f1cc4955a0ca99d588344322`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:29.898004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-19141249-e687-4535-a957-a114080f27f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:29.917645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 833/1647] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:29.935506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:29.966876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae799d-6951-4aab-b4bc-128a829343a2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:29.973836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:29.974539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:29.974831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:29.975211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:29.994751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:29.996548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476cd4f-6eb0-4b6f-89b2-fc724f952c2a service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:30.020457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 815/1648] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:31.257418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d4bde-7165-46e1-8b72-099f045deebc None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:31.258452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d4bde-7165-46e1-8b72-099f045deebc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:31.258948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d4bde-7165-46e1-8b72-099f045deebc None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:31.260728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 834/1649] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 25 16:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:24:31.283786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:31.285563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:31.286759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:31.447643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:31.452675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:31.639586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 816/1650] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:24:31.673738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d4bde-7165-46e1-8b72-099f045deebc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:31.704485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d4bde-7165-46e1-8b72-099f045deebc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:31.729944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:31.730677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:31.730961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:31.731240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:31.784625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:31.795157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:31.819843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 835/1651] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:33.690090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:34.083934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:34.100290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:34.263260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3e3c-e231-45fd-a8ba-4f809606f51c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:34.314947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76959a8-5e9c-4700-bc8e-2247ed470aca service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:34.321824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:34.323465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:34.324000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:34.329252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:34.332861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:34.333664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:34.334421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] PATH_INFO: `/v3/users/716072978f3948419a06e14ef4746de8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:34.337078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=716072978f3948419a06e14ef4746de8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:34.346401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] PATH_INFO: `/v3/users/3a30c39e01734a7c8630e9ed28cbf510` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:34.348202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:34.351934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:34.357521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a30c39e01734a7c8630e9ed28cbf510)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:34.359650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:34.359650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 25 16:24:34.359650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: Jul 25 16:24:34.393792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:34.582752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716072978f3948419a06e14ef4746de8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.634355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716072978f3948419a06e14ef4746de8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.635518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716072978f3948419a06e14ef4746de8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.677327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716072978f3948419a06e14ef4746de8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.678453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ea084617-d581-4228-86f1-eac59423ab3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716072978f3948419a06e14ef4746de8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.746375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a30c39e01734a7c8630e9ed28cbf510'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.774515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 836/1652] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:34 2022] DELETE /identity/v3/users/716072978f3948419a06e14ef4746de8 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:24:34.790619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087d28-41d1-4385-b90e-f245cf377f14 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:34.792766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087d28-41d1-4385-b90e-f245cf377f14 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:34.793660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98087d28-41d1-4385-b90e-f245cf377f14 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:34.797675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 837/1653] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 25 16:24:34 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:34.800403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a30c39e01734a7c8630e9ed28cbf510'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.800613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:34.804535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:34.806108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:34.826673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a30c39e01734a7c8630e9ed28cbf510'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.893131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a30c39e01734a7c8630e9ed28cbf510'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.896159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-30021aab-fd5e-455e-9794-7d3fd4a54607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a30c39e01734a7c8630e9ed28cbf510'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:34.901545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:34.904690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:34.916467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 817/1654] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:33 2022] DELETE /identity/v3/users/3a30c39e01734a7c8630e9ed28cbf510 => generated 0 bytes in 1227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:24:34.919722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbaaa-6fe3-47e0-8dbe-a2028b1f156a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:34.920305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbaaa-6fe3-47e0-8dbe-a2028b1f156a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:34.920835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bbaaa-6fe3-47e0-8dbe-a2028b1f156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:34.964451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e0bbaaa-6fe3-47e0-8dbe-a2028b1f156a None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:34.966821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e0bbaaa-6fe3-47e0-8dbe-a2028b1f156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:35.104166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 818/1655] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 25 16:24:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:35.106545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:35.106835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:35.107116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:35.170638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:35.172039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:35.214601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 838/1656] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:35.217050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:35.259632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5942c-9448-4a89-9304-faccedfc6d45 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:35.279242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:35.280820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:35.281877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:35.282583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:35.293248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:35.337574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:35.352125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:35.377631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 819/1657] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:35.377631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:35.379366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:35.409931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:35.416694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 839/1658] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:35.433247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:35.441690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:35.554748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021da4-0303-4771-82e6-aa0f7b1da285 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:35.577160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:35.578124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:35.578410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:35.578695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:35.606746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:35.608282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:35.639266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 820/1659] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:35.687511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf09af-9c66-4709-ad85-5378b2bcbd0b service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:35.705175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:35.706374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:35.706874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:35.707358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:35.770957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:35.774041 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556733e-f923-41bc-93b6-eaf5fb87b2dd service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:35.832395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 840/1660] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:37.393892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:37.399207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:37.460679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccfa31-d563-4337-9bda-a43cc9e49424 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:37.474827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:37.476157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:37.478671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:37.480736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] PATH_INFO: `/v3/projects/f57d34a39f87447f82b90b67a29cf2c8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:37.488624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57d34a39f87447f82b90b67a29cf2c8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:37.495556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:37.743425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57d34a39f87447f82b90b67a29cf2c8', 'request_id': 'req-db7885f1-67c8-4580-b5bc-96ff96a0ce93'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:37.767647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57d34a39f87447f82b90b67a29cf2c8', 'request_id': 'req-db7885f1-67c8-4580-b5bc-96ff96a0ce93'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:37.781067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57d34a39f87447f82b90b67a29cf2c8', 'request_id': 'req-db7885f1-67c8-4580-b5bc-96ff96a0ce93'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:37.834494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] The token cache is being invalidate because project f57d34a39f87447f82b90b67a29cf2c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:37.837310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:37.838459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-db7885f1-67c8-4580-b5bc-96ff96a0ce93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:37.848790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 821/1661] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:37 2022] DELETE /identity/v3/projects/f57d34a39f87447f82b90b67a29cf2c8 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:37.861798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:37.861798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:37.861798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:37.934960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:37.937083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:37.953605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:38.168248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 841/1662] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:38.182924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:38.183364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:38.183792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:38.249292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:38.251406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:38.295190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 822/1663] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:38.304357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:38.346726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef4378-8181-44c7-823e-1ec043efb77c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:38.378758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:38.381177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:38.381749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:38.384988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:38.389187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:38.392915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:38.407803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:38.493214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 842/1664] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:38.497986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:38.532321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819ceb7-b50f-4798-b331-95599025d18b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:38.566307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:38.572664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:38.572664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:38.572664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:38.572664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:38.576408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a630fe-72b1-45a0-83bd-570b3bc9e8c6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:38.733746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 823/1665] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:38 2022] POST /identity/v3/projects => generated 403 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:38.738617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:38.767408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84d763-8410-4f6a-b377-ee496f11c575 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:38.836578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:38.836578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:38.836578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:38.945287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] MFA Rules not processed for user `644c0a532fa747c7abf4d53759e89181`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:38.971710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.067204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:39.073853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.074208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.077645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.085241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:39.096862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:39.162066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:39.184592 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 824/1666] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:39.192833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:39.196877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.209780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 843/1667] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:38 2022] POST /identity/v3/users => generated 302 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:39.214595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:39.215664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4943416-7e29-4e8a-9780-26f575321400 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:39.221787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.226707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:39.227802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.228122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.228406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] PATH_INFO: `/v3/users/ed49dd65d33b408a8d4850c30e4e8a25` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.230388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed49dd65d33b408a8d4850c30e4e8a25)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:39.232845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:39.457694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed49dd65d33b408a8d4850c30e4e8a25'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:39.537631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed49dd65d33b408a8d4850c30e4e8a25'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:39.575511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed49dd65d33b408a8d4850c30e4e8a25'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:39.613270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ad896-d57c-4c04-87e9-c39d36f5a21f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:39.632858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:39.634060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.634644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.635048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.652092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed49dd65d33b408a8d4850c30e4e8a25'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:39.653112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed49dd65d33b408a8d4850c30e4e8a25'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:39.674291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:39.677152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e957bc-826b-4bfe-b9df-2b193a45ddb1 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:39.679973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 825/1668] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:39 2022] DELETE /identity/v3/users/ed49dd65d33b408a8d4850c30e4e8a25 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:39.681083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:39.685284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.713183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 844/1669] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:39.715981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.716418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.716786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.857598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:39.873067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.882421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b439f-92de-410d-8474-d4a063e03602 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:39.901191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:39.901191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.901191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.901191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.901665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:39.901665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:39.929622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 826/1670] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:39.930797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:39.940206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 845/1671] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:39.941078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.942273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:39.944815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:39.967095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee070-8602-4428-b887-7feeabcfc99f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:39.980622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:39.984746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:39.985108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:39.985367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] PATH_INFO: `/v3/projects/157fa0d8c8d04d099ce05b715300e22b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:39.986868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:39.991972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=157fa0d8c8d04d099ce05b715300e22b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:39.994830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:40.003812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3606e71-69b2-49b5-b9d6-a23e55c88126 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:40.032904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:40.034957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:40.035734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:40.036407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] PATH_INFO: `/v3/users/1b025bd914a34faf900b460eaedc1493` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:40.044049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b025bd914a34faf900b460eaedc1493)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:40.052610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:40.140992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157fa0d8c8d04d099ce05b715300e22b', 'request_id': 'req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.154812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157fa0d8c8d04d099ce05b715300e22b', 'request_id': 'req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.186062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:40.217140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157fa0d8c8d04d099ce05b715300e22b', 'request_id': 'req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.243289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b025bd914a34faf900b460eaedc1493'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.274720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b025bd914a34faf900b460eaedc1493'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.277587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] The token cache is being invalidate because project 157fa0d8c8d04d099ce05b715300e22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:40.278070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b025bd914a34faf900b460eaedc1493'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.278604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.282294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.290825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 827/1672] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:39 2022] DELETE /identity/v3/projects/157fa0d8c8d04d099ce05b715300e22b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:40.292971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:40.300089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b025bd914a34faf900b460eaedc1493'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.301174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b025bd914a34faf900b460eaedc1493'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.302242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:40.367574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 846/1673] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:39 2022] DELETE /identity/v3/users/1b025bd914a34faf900b460eaedc1493 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:40.369463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:40.377643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:40.499895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad40c-ea3e-41e9-bb55-6dc04a02e87e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:40.652394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fdfb1-1915-494d-9f3a-91cdbc7eb321 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:40.682564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:40.693438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:40.696177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:40.702819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] PATH_INFO: `/v3/users/0af1812a4a694689ad9fbf038815a1a6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:40.711081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af1812a4a694689ad9fbf038815a1a6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:40.718934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:40.889574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:40.891877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:40.892735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:40.893427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:40.895132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:40.898235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e5352-3b18-4dee-b4cc-c61af8e46459 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:40.931266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af1812a4a694689ad9fbf038815a1a6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.967077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af1812a4a694689ad9fbf038815a1a6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.973096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af1812a4a694689ad9fbf038815a1a6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.973182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 828/1674] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 677 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:40.977472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:40.977954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:40.978415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:40.996604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af1812a4a694689ad9fbf038815a1a6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:40.998078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ed4cea10-07ad-4ef7-89cd-68729faf9b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af1812a4a694689ad9fbf038815a1a6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:41.044599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:41.068983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:41.070376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 847/1675] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:40 2022] DELETE /identity/v3/users/0af1812a4a694689ad9fbf038815a1a6 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:41.072750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:41.073258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:41.078118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:41.087415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:41.246644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:41.248809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:41.412506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 829/1676] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:41.417027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:41.422090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:41.552220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:41.596583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 848/1677] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:41.598153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:41.603795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:41.631299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c628ffd-e679-4bc5-8cce-460951727871 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:41.651322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:41.651748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:41.652033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:41.652404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:41.660673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:41.671303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20128db-a942-4940-80b7-53abf8908dc1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:41.697978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:41.699938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:41.702730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:41.704410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:41.704903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:41.726067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] PATH_INFO: `/v3/projects/2b09c8b83f87428cbb13ca13db2caf84/users/ee65481300f340b780faa47026464afe/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:41.757622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 830/1678] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:41.758605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:41.778760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c05b-c743-4e34-bd3b-900c345e224e service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:41.798235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:41.799087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:41.799300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:41.799630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:41.829615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:41.831334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23d019-59db-4bca-9ffd-466108782dc7 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:41.854772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b09c8b83f87428cbb13ca13db2caf84, user_id=ee65481300f340b780faa47026464afe, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:41.864399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 831/1679] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:41.869433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:41.945614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 849/1680] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:41 2022] PUT /identity/v3/projects/2b09c8b83f87428cbb13ca13db2caf84/users/ee65481300f340b780faa47026464afe/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:41.958942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a1f6b-039d-487a-8c0f-23181f71229b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:41.959500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a1f6b-039d-487a-8c0f-23181f71229b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:41.959673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a1f6b-039d-487a-8c0f-23181f71229b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:42.012024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:42.014717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:42.062831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-151a1f6b-039d-487a-8c0f-23181f71229b None None] MFA Rules not processed for user `ee65481300f340b780faa47026464afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:42.064130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-151a1f6b-039d-487a-8c0f-23181f71229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:42.231642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 832/1681] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:42.233733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:42.234515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:42.234515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:42.280062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] MFA Rules not processed for user `2d1eb89923784f1c83d2077c6b6926e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:42.282037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:42.392920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 833/1682] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:42.404566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:42.516303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:42.523835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d60e-dade-4e87-834a-289c31c87bc9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:42.558136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:42.562587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:42.563847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:42.568055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:42.577731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468c71-aca7-42b1-8edf-85aea1f30a68 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:42.587397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:42.588536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:42.589730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:42.590214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:42.593794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:42.596212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e50e5-b5ba-4ab3-855e-74ca670e5dd2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:42.683979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 834/1683] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:42 2022] POST /identity/v3/projects => generated 403 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:42.713849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:42.714710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:42.715242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:42.727817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:42.786820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] MFA Rules not processed for user `2d1eb89923784f1c83d2077c6b6926e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:42.788144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:42.881491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:42.886129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:42.942564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 850/1684] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:24:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 931 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:24:42.945570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:42.971215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 835/1685] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:42.999981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:43.001901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:43.034108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67aa21-84be-4839-b1c1-603872476c40 service swift] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:43.246550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14111c6-db7c-49a7-9d8c-8b9e99a962c8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:43.283383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:43.284118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:43.284410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:43.284790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:43.328310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:43.337993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:43.389461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 836/1686] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:43.390629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:43.410011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:43.556748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:43.562102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:43.562969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:43.563602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:43.585761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:43.602512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:43.702956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 851/1687] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:42 2022] POST /identity/v3/users => generated 308 bytes in 760 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:43.711963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:43.715282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:43.739287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458455b-aec9-4701-aa5b-39295a316c53 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:43.767135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e458724-2b21-4904-908a-7c97d92bb99f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:43.767975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:43.773857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:43.775216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:43.779897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] PATH_INFO: `/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:43.785572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10df4de2a8e94cd6a95adf7e42cd8a38)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:43.787826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:43.789034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:43.789512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:43.792883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:43.797103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:43.798795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:43.802300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:43.865613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 852/1688] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:43.867674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:43.917823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50406d19-a280-46c6-b0ad-fa543ac6e9b1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:43.942547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:43.944710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:43.946232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:43.946803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] PATH_INFO: `/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d/users/d3580b47295f4eb59c657e3dcec9198d/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.021456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53d7aeb77ad4f53a7cbdd1d6803249d, user_id=d3580b47295f4eb59c657e3dcec9198d, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:44.030046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:44.049775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10df4de2a8e94cd6a95adf7e42cd8a38', 'request_id': 'req-01eb5494-ff75-4063-833e-aa01095de454'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:44.105148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10df4de2a8e94cd6a95adf7e42cd8a38', 'request_id': 'req-01eb5494-ff75-4063-833e-aa01095de454'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:44.120602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 853/1689] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:43 2022] PUT /identity/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d/users/d3580b47295f4eb59c657e3dcec9198d/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:44.130269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:44.178290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10df4de2a8e94cd6a95adf7e42cd8a38', 'request_id': 'req-01eb5494-ff75-4063-833e-aa01095de454'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:44.290414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] The token cache is being invalidate because project 10df4de2a8e94cd6a95adf7e42cd8a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:44.291055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:44.293630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-01eb5494-ff75-4063-833e-aa01095de454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:44.310143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 837/1690] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:43 2022] DELETE /identity/v3/projects/10df4de2a8e94cd6a95adf7e42cd8a38 => generated 0 bytes in 920 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:44.316096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736b09e-1a41-44b9-aefb-2013c8cf4743 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:44.316295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736b09e-1a41-44b9-aefb-2013c8cf4743 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:44.316560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736b09e-1a41-44b9-aefb-2013c8cf4743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.397384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8736b09e-1a41-44b9-aefb-2013c8cf4743 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:44.398489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8736b09e-1a41-44b9-aefb-2013c8cf4743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:44.572136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1ba-c6d3-494f-ad97-b8756064d577 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:44.612211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:44.613534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:44.614052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:44.615425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.616567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:44.631651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:44.650738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 838/1691] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:24:44.654395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:44.654728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:44.654965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.696184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 854/1692] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 570 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:24:44.719265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:44.726018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:44.726268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:44.729070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:44.788306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad70d14-6b4c-4dd1-b7f0-43273ad11578 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:44.798395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 839/1693] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:44.801770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:44.803091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:44.804153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:44.804442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:44.804722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] PATH_INFO: `/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d/users/d3580b47295f4eb59c657e3dcec9198d/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.810161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53d7aeb77ad4f53a7cbdd1d6803249d, user_id=d3580b47295f4eb59c657e3dcec9198d, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:44.826236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ae79-19fd-4d75-b875-1acd1a02acab admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:44.839292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588c757-55f3-4c01-a944-d68a9b074c89 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:44.880101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 855/1694] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:44 2022] PUT /identity/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d/users/d3580b47295f4eb59c657e3dcec9198d/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:44.893540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:44.894347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:44.894894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:44.947844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:45.139347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] MFA Rules not processed for user `d3580b47295f4eb59c657e3dcec9198d`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:45.147025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:45.272392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:45.276991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:45.278608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:45.282582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:45.287627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:45.289228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:45.296773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:45.380570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 840/1695] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 572 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:45.384099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:45.399264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 856/1696] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:45.412101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:45.421964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:45.431128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec47e2-e72f-4688-9207-5f5c9ae6b20f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:45.495680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:45.502654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:45.509771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:45.517515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:45.518490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:45.526129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:45.589273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:45.666093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975fe46-0ff6-4e42-9cf6-3de5cb79274d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:45.681733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:45.682485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:45.682775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:45.683063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:45.714611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:45.716621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:45.726777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 841/1697] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:45 2022] POST /identity/v3/projects => generated 369 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:45.739841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:45.754441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 857/1698] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:46.111125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583f13-7fc1-4fc5-81ec-921e1c744d42 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:46.139644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:46.140559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:46.140862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:46.141147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:46.143305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:46.144403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:46.273925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 842/1699] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:45 2022] POST /identity/v3/users => generated 285 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:46.283764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:46.372917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98f48-d765-4010-8ec5-c92384ccde7c service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:46.418687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:46.425715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:46.429125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:46.432425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:46.455697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:46.462220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:46.511931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:46.530749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d330c-59e9-4be8-bc57-ae9363b31893 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:46.549601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 858/1700] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:46.552444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:46.556015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:46.601997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:46.606370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b157d8c-c3d2-4d0a-a947-53c904dba2e3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:46.615938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:46.616309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:46.616605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] PATH_INFO: `/v3/users/ee65481300f340b780faa47026464afe` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:46.622396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee65481300f340b780faa47026464afe)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:46.627826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d851-4322-4093-b348-db205315914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:46.632150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d851-4322-4093-b348-db205315914a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:46.632664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d851-4322-4093-b348-db205315914a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:46.633152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d851-4322-4093-b348-db205315914a admin admin] PATH_INFO: `/v3/projects/13beb0177fdd4ba9b43a260045fce37e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:46.638299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d851-4322-4093-b348-db205315914a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13beb0177fdd4ba9b43a260045fce37e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:46.639089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:46.641755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-60d1d851-4322-4093-b348-db205315914a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 16:24:46.641755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 25 16:24:46.641755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: Jul 25 16:24:46.645802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d851-4322-4093-b348-db205315914a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:46.872313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee65481300f340b780faa47026464afe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:46.919258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee65481300f340b780faa47026464afe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:46.937531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13beb0177fdd4ba9b43a260045fce37e', 'request_id': 'req-60d1d851-4322-4093-b348-db205315914a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:46.950026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee65481300f340b780faa47026464afe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:46.959748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13beb0177fdd4ba9b43a260045fce37e', 'request_id': 'req-60d1d851-4322-4093-b348-db205315914a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.009207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee65481300f340b780faa47026464afe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.013912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee65481300f340b780faa47026464afe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.029488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13beb0177fdd4ba9b43a260045fce37e', 'request_id': 'req-60d1d851-4322-4093-b348-db205315914a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.055605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 843/1701] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:46 2022] DELETE /identity/v3/users/ee65481300f340b780faa47026464afe => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:24:47.060156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:47.065074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:47.094168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] The token cache is being invalidate because project 13beb0177fdd4ba9b43a260045fce37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:47.094478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.101590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:47.110109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 859/1702] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:46 2022] DELETE /identity/v3/projects/13beb0177fdd4ba9b43a260045fce37e => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:47.115266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:47.142319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:47.155674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:47.414910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:47.480238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2579bf1-e861-47d0-90fe-053e121bafc6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:47.528914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:47.529751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:47.530045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:47.530380 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:47.531215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:47.536874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa00ff-7901-4e32-b1ee-8e3e1392c678 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:47.586252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 844/1703] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 530 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:47.589029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb7cb-cfcb-4639-b02d-abc68bf3d3d4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:47.594471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb7cb-cfcb-4639-b02d-abc68bf3d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:47.596496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb7cb-cfcb-4639-b02d-abc68bf3d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:47.778801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d851-4322-4093-b348-db205315914a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:47.792013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b99eb7cb-cfcb-4639-b02d-abc68bf3d3d4 None None] MFA Rules not processed for user `2d1eb89923784f1c83d2077c6b6926e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:47.804145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b99eb7cb-cfcb-4639-b02d-abc68bf3d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:47.819812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:47.825895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:47.833276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:47.834855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] PATH_INFO: `/v3/projects/0ab28729cacc4da1a5025f52ac5f1127` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:47.859306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab28729cacc4da1a5025f52ac5f1127)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:47.921578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:48.221892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 845/1704] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 635 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:48.231890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:48.235265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:48.237517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:48.297022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab28729cacc4da1a5025f52ac5f1127', 'request_id': 'req-ff2bb3fe-829a-40d1-9660-4f161bf9f352'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:48.358592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab28729cacc4da1a5025f52ac5f1127', 'request_id': 'req-ff2bb3fe-829a-40d1-9660-4f161bf9f352'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:48.415182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:48.418216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:48.420299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab28729cacc4da1a5025f52ac5f1127', 'request_id': 'req-ff2bb3fe-829a-40d1-9660-4f161bf9f352'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:48.479804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] The token cache is being invalidate because project 0ab28729cacc4da1a5025f52ac5f1127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:48.480478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:48.484653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ff2bb3fe-829a-40d1-9660-4f161bf9f352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:48.487501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 860/1705] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:47 2022] DELETE /identity/v3/projects/0ab28729cacc4da1a5025f52ac5f1127 => generated 0 bytes in 1377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:48.502141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a75a-7ee3-4353-832f-fc173cd17bc1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:48.502834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a75a-7ee3-4353-832f-fc173cd17bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:48.503439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a75a-7ee3-4353-832f-fc173cd17bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:48.589075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4487a75a-7ee3-4353-832f-fc173cd17bc1 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:48.601275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4487a75a-7ee3-4353-832f-fc173cd17bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:48.706891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 846/1706] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:48.708290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:48.724700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:48.789142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f320990-e0db-4065-8377-2b0e527135a9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:48.808743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:48.814758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:48.816654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:48.818648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] PATH_INFO: `/v3/projects/a539995e57a64c149016227046bd4446/users/bc7b85321d984136b2f6c0d771608460/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:48.906635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a539995e57a64c149016227046bd4446, user_id=bc7b85321d984136b2f6c0d771608460, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:48.915637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa76e-763e-427e-b5ab-32814e99ce80 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:48.961576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 847/1707] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:48 2022] PUT /identity/v3/projects/a539995e57a64c149016227046bd4446/users/bc7b85321d984136b2f6c0d771608460/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:48.969599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:48.970446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:48.971384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:48.992293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 861/1708] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:48.995793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b828bab-f15d-4ee2-a112-ee19a1aae7d4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:48.996492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b828bab-f15d-4ee2-a112-ee19a1aae7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:48.997033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b828bab-f15d-4ee2-a112-ee19a1aae7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:49.058975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:49.063290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:49.071872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5b828bab-f15d-4ee2-a112-ee19a1aae7d4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:49.083321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5b828bab-f15d-4ee2-a112-ee19a1aae7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:49.310599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:49.442843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 848/1709] 10.222.0.102 () {58 vars in 1016 bytes} [Mon Jul 25 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:49.444229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:49.455047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:49.487948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f5b80-6d56-407a-88cc-06e2d206e47e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:49.509593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:49.510621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:49.511182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:49.511779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:49.520904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 862/1710] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:49.522569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021dea-71f0-4915-93c3-7cfef2d541dc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:49.522854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021dea-71f0-4915-93c3-7cfef2d541dc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:49.523137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56021dea-71f0-4915-93c3-7cfef2d541dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:49.559141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:49.596781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16edf5-28a1-4087-9ab5-42afe6411184 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:49.681992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 849/1711] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:49.688077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:49.691309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-56021dea-71f0-4915-93c3-7cfef2d541dc None None] MFA Rules not processed for user `bc7b85321d984136b2f6c0d771608460`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:49.692774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-56021dea-71f0-4915-93c3-7cfef2d541dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:49.698789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:49.702486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:49.819074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:49.845018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:49.851893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:49.916027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 850/1712] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:49.924096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:49.988603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 863/1713] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:49.996969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.001595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a54c1-30a1-4636-8861-7fcb64ae4dd9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.001887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.005823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.026594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.030868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.032831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.035452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.085274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.097870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:50.098805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:50.101803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.158417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 864/1714] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:50.158417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.161115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 851/1715] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:50.166897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.223637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a4162-7c64-43f1-a4f8-08c97221f5e1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.260570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.267991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.268454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.270287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.271039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:50.271360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.276409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.279117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740b95-9b89-443f-8cad-02deeca983c5 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.302674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.307036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.310713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.311895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.317828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.325672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.337160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 865/1716] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:50.340883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.372618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c8dc-29ec-42dc-91be-21c1cf712a4c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.424534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.425717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 852/1717] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:50 2022] POST /identity/v3/projects => generated 369 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:50.428810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.429667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.432188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.438739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.444547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:50.448093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.461712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.598172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 866/1718] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:50.600217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.637480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c8455-8140-4e2f-9e86-50c46d19df8a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.671104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9898b-6b84-4206-bab6-69fe2c6ef244 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:50.685637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.687727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.688465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.689112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.695383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.698263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.700683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:50.702637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:50.703872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:50.703872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:50.705503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:50.716436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:50.790709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 853/1719] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:50 2022] POST /identity/v3/projects => generated 381 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:50.795455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.799097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 867/1720] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:50 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:50.800463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:50.876810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1202ba-df3f-4207-8c9c-01ee73965ab1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.006917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74010bbc-8295-4c3f-8155-fe5b09e74ed1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.030141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.032256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.035518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.036250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.037850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.048231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.191442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 854/1721] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:50 2022] POST /identity/v3/projects => generated 363 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:51.201602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:51.210927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.212340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.212823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.213291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.215094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.229687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.323835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 868/1722] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:50 2022] POST /identity/v3/users => generated 291 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:51.325788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:51.342259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:51.363253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1906b1-c667-4ba7-977e-3a8d2e258131 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.513107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:51.652209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c1328-af95-413c-81ce-d86b58be091c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.661972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.662767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.663235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.666912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.674975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.679245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.679988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.680248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.683411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.686699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.693379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.701676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.758552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 855/1723] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 566 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:51.761041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:51.785745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 869/1724] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:51 2022] POST /identity/v3/users => generated 282 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:51.787877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:51.792217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359f97b-ab8b-4a33-9e06-216e7f465e6c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.812388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.815588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.815941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.816242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.819765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.823207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.852617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862791d-dbc0-489d-b6f8-4c9e318273da admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.859229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 856/1725] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:51.862358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:51.898626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.903809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.907656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.907656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] PATH_INFO: `/v3/projects/e9765902d8c04c638916620be0c9231a/users/1f289eee117d40ffb607d24cc6c565de/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.910852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793093d-8c1a-4de9-8b2b-a535869bcc9a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:51.930056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:51.930803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:51.931244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:51.931532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:51.932317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:51.940583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:51.985980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 857/1726] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:51.986415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.017006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c570-c526-4caa-bd57-fdc678b1d329 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:52.021281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9765902d8c04c638916620be0c9231a, user_id=1f289eee117d40ffb607d24cc6c565de, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:52.032800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:52.044825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:52.046197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:52.046513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:52.049011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:52.050919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:52.056808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:52.091304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 870/1727] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:51 2022] PUT /identity/v3/projects/e9765902d8c04c638916620be0c9231a/users/1f289eee117d40ffb607d24cc6c565de/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:52.093454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.099981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 858/1728] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:52.102729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.152045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5430-11d7-4604-a8da-5823d37e4b06 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:52.321396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fa52a-6632-45a1-b945-3fc4599bba5d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:52.330242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:52.345932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:52.347704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:52.348480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:52.351643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:52.355069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:52.413304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 871/1729] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:52.416264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.454745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b0b6-204c-4aa4-95e8-5acc97dc5b98 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:52.474267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:52.475934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:52.476426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:52.476941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] PATH_INFO: `/v3/projects/c28e22852a274c6fa26da14da20f47e4/users/961d468934304d73aa7865dcb53a20c1/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:52.524531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:52.525237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:52.525680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:52.525968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:52.527374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:52.527565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28e22852a274c6fa26da14da20f47e4, user_id=961d468934304d73aa7865dcb53a20c1, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:52.533152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:52.542104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:52.587243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 872/1730] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:52 2022] PUT /identity/v3/projects/c28e22852a274c6fa26da14da20f47e4/users/961d468934304d73aa7865dcb53a20c1/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:52.589781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.645385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 859/1731] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:52.647218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:52.656075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:52.717723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4c89-6de7-421e-9f18-867fa98238f5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:52.842732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:52.978802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53cf3-670d-4622-9701-036749d7fa27 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:53.004359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:53.008139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.012630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.013634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] PATH_INFO: `/v3/projects/0b1c8c7afd664267ae53664f5747a98d/users/590940ccd5374abea292d1e4d6c94e87/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.026638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:53.027640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.028588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.028895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] PATH_INFO: `/v3/projects/2b09c8b83f87428cbb13ca13db2caf84` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.049835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b09c8b83f87428cbb13ca13db2caf84)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:53.051882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:53.105996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1c8c7afd664267ae53664f5747a98d, user_id=590940ccd5374abea292d1e4d6c94e87, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:53.145137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d04d0d-b50a-4805-81dd-ec26f38785b1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:53.203849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:53.225523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 873/1732] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:52 2022] PUT /identity/v3/projects/0b1c8c7afd664267ae53664f5747a98d/users/590940ccd5374abea292d1e4d6c94e87/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:53.232323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df1e0-4420-4315-8dc1-130f44f01ede None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.233992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df1e0-4420-4315-8dc1-130f44f01ede None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.235274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df1e0-4420-4315-8dc1-130f44f01ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.255147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b09c8b83f87428cbb13ca13db2caf84', 'request_id': 'req-3466f90a-715d-4c82-bfca-9b0539b6e222'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:53.285168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b09c8b83f87428cbb13ca13db2caf84', 'request_id': 'req-3466f90a-715d-4c82-bfca-9b0539b6e222'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:53.326498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b09c8b83f87428cbb13ca13db2caf84', 'request_id': 'req-3466f90a-715d-4c82-bfca-9b0539b6e222'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:53.379973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] The token cache is being invalidate because project 2b09c8b83f87428cbb13ca13db2caf84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:53.380411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:53.388044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:53.391043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d24df1e0-4420-4315-8dc1-130f44f01ede None None] MFA Rules not processed for user `961d468934304d73aa7865dcb53a20c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:53.392091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 860/1733] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:52 2022] DELETE /identity/v3/projects/2b09c8b83f87428cbb13ca13db2caf84 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:53.396662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:53.402886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d24df1e0-4420-4315-8dc1-130f44f01ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:53.406643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:53.639459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466f90a-715d-4c82-bfca-9b0539b6e222 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:53.648929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:53.650527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.651123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.651551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] PATH_INFO: `/v3/projects/e9765902d8c04c638916620be0c9231a/users/1f289eee117d40ffb607d24cc6c565de/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.654351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 874/1734] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:53.655183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9765902d8c04c638916620be0c9231a, user_id=1f289eee117d40ffb607d24cc6c565de, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:53.656857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.657379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.660381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:53.665272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.688895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 861/1735] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:53 2022] PUT /identity/v3/projects/e9765902d8c04c638916620be0c9231a/users/1f289eee117d40ffb607d24cc6c565de/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:53.690464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:53.692431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:53.811598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] MFA Rules not processed for user `590940ccd5374abea292d1e4d6c94e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:53.826435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:53.936969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8362-c160-471a-9029-20edc43f61b4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:53.949644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:53.952649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:53.953577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:53.954639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] PATH_INFO: `/v3/users/d3580b47295f4eb59c657e3dcec9198d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:53.959163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3580b47295f4eb59c657e3dcec9198d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:53.964827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:54.107641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3580b47295f4eb59c657e3dcec9198d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:54.110620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 875/1736] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:54.117270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:54.117727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:54.144177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3580b47295f4eb59c657e3dcec9198d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:54.164259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:54.175986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3580b47295f4eb59c657e3dcec9198d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:54.182655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f710c6-dc70-4c26-aeab-580da730fb74 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:54.203829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:54.205025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:54.205553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:54.206948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:54.210267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:54.222154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:54.231257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3580b47295f4eb59c657e3dcec9198d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:54.234945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5e76f06d-842e-455b-b4cd-8c51e2f0d79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3580b47295f4eb59c657e3dcec9198d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:54.261733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 862/1737] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:24:53 2022] DELETE /identity/v3/users/d3580b47295f4eb59c657e3dcec9198d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:54.268756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:54.269221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:54.269782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:54.341788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 876/1738] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:54 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:54.343498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:54.349361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:54.390565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] MFA Rules not processed for user `1f289eee117d40ffb607d24cc6c565de`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:54.393701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:54.441540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:54.705318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60184849-9c79-4aca-b088-9e7fe2dd7950 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:54.742877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:54.744507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:54.745329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:54.746060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:54.747532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:54.758436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20036c18-48f7-4638-b613-1330bbe20ccd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:54.851897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 863/1739] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:54.851897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:54.861087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:54.895746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 877/1740] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:54.898320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:54.899743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:54.899743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:54.903245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 878/1741] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:24:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:24:54.906163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:54.909827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:54.911580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1dc52-5b3a-423d-b749-15923e25f08c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.188051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.189118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.189118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.189435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.190113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.192124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.202760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d6d3-2337-4e9b-a20b-c4339e5b50e2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.217657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.219016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.219633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.220253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.258182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 864/1742] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:54 2022] POST /identity/v3/users => generated 296 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:55.258759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.305915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268716e-dd5c-4ab0-984b-dc1f88f60f46 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.328665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.329566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.329853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.330265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.331026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.340646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd6649-34f0-48a4-9f03-0fd6bbd72ec1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.404905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 865/1743] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:55 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:55.417132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.417945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.418418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.474504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:55.477166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.478883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.480611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:55.498959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 879/1744] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:55.500766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.517076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2235-1505-497f-85ae-29dd83d81307 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.524833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.525971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.526303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.526662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.527897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.530378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.558672 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 880/1745] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:55.559691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.584081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7657f0-917f-41a9-957f-25069dab2353 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.585745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:55.606794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.607632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.607924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.608333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.613844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.615914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.651428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 881/1746] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:55.652405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.697931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad8adb-b249-4333-826d-e805e8429fb8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.731235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.735059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 866/1747] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:55.735934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.737582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.738632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.741743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] PATH_INFO: `/v3/projects/d110540babfd41c69bbabcb0673d5463/users/33a480f2ed4d4fefba1f65cdf1fcaec3/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.802715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5a458-7e30-40b1-88b0-45977eca8484 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:55.808308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d110540babfd41c69bbabcb0673d5463, user_id=33a480f2ed4d4fefba1f65cdf1fcaec3, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.811827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.822765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:55.823988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:55.824433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:55.824864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] PATH_INFO: `/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12/users/dd90c7bcd8c646acaf371acee1ad0f79/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:55.861105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 882/1748] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:55 2022] PUT /identity/v3/projects/d110540babfd41c69bbabcb0673d5463/users/33a480f2ed4d4fefba1f65cdf1fcaec3/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:55.862520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee26e5c53e5b40de92f8cee9a40ddc12, user_id=dd90c7bcd8c646acaf371acee1ad0f79, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:55.869535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:55.871879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:55.924508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:55.932337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 867/1749] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:55 2022] PUT /identity/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12/users/dd90c7bcd8c646acaf371acee1ad0f79/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:55.934144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:56.214013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f731f-9046-4447-bc98-eadb6245b134 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:56.239759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:56.240417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.241014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.241295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.250054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:56.251982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:56.256648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879fcd2-67e4-45ff-9d36-93300433fa8e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:56.290272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:56.295342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.295735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.296061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.296939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:56.308759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:56.316111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 883/1750] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:56.318847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:56.365397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 868/1751] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:56.371188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:56.373099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082d0c-073f-4510-8b73-188831fa3ea9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:56.425656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:56.428034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.428235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.428526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] PATH_INFO: `/v3/projects/d110540babfd41c69bbabcb0673d5463/users/33a480f2ed4d4fefba1f65cdf1fcaec3/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.436654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d110540babfd41c69bbabcb0673d5463, user_id=33a480f2ed4d4fefba1f65cdf1fcaec3, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:56.440570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a8558-8925-4206-823a-7b4eedd2d83e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:56.446348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1c3d8-266b-4320-ae94-a845a3201e8c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:56.463318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:56.468843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.474195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.477060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] PATH_INFO: `/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12/users/dd90c7bcd8c646acaf371acee1ad0f79/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.490097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee26e5c53e5b40de92f8cee9a40ddc12, user_id=dd90c7bcd8c646acaf371acee1ad0f79, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:56.500520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 884/1752] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:56 2022] PUT /identity/v3/projects/d110540babfd41c69bbabcb0673d5463/users/33a480f2ed4d4fefba1f65cdf1fcaec3/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:56.501730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbcf2a-6c0f-4189-a507-fa949c1c69f1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:56.505539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.505843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.506127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.523593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 869/1753] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:56 2022] PUT /identity/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12/users/dd90c7bcd8c646acaf371acee1ad0f79/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:56.534584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:56.545389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:56.547043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:56.598476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] MFA Rules not processed for user `dd90c7bcd8c646acaf371acee1ad0f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:56.600813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:56.602802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] MFA Rules not processed for user `33a480f2ed4d4fefba1f65cdf1fcaec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:56.604656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:56.854127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 885/1754] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:56.854127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:56.864836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:56.888564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 870/1755] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:56.889646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:56.910555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:56.940718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:57.051449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74ac1b-7dda-4c2d-8861-3e6073d501be None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:57.066998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:57.070128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:57.071374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:57.072488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:57.085384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:57.087932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:57.110905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 871/1756] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:57.112352 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:57.132001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24540b23-c661-4bfc-9302-3e94d619a419 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:57.138520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:57.139704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:57.140168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:57.140569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:57.156809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:57.159442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:57.178230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 872/1757] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:57.212126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ac252-e072-4f9c-90bd-c02c02f16a77 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:57.239009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:57.239848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:57.240229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:57.240441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:57.297995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:57.305707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:57.353480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 886/1758] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:57.486721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:57.491033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:57.644933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:57.654389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:57.696345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf2e5-92a0-4c37-90d1-0ebd12b44217 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:57.718193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:57.719996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:57.720676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:57.723437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:57.729199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:57.949761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9a266-65b5-414d-ac0b-5af84520e029 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:57.999435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:58.001843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.009065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.009988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] PATH_INFO: `/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.023365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53d7aeb77ad4f53a7cbdd1d6803249d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:58.035040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:58.040351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:58.043738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:58.080452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 887/1759] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:58.190515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:58.253447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae12cd-d668-4f45-9add-b38bae77fcec service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:58.261525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53d7aeb77ad4f53a7cbdd1d6803249d', 'request_id': 'req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:58.277231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53d7aeb77ad4f53a7cbdd1d6803249d', 'request_id': 'req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:58.293216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53d7aeb77ad4f53a7cbdd1d6803249d', 'request_id': 'req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:58.314733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] The token cache is being invalidate because project f53d7aeb77ad4f53a7cbdd1d6803249d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:24:58.315313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:58.317828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04c19a22-8e3f-444e-99b3-6c30c4c5f9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:24:58.320294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 873/1760] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:24:57 2022] DELETE /identity/v3/projects/f53d7aeb77ad4f53a7cbdd1d6803249d => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:58.327773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77881-413c-4dc7-ba87-89551e0bf938 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.328360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77881-413c-4dc7-ba87-89551e0bf938 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.328855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77881-413c-4dc7-ba87-89551e0bf938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.375022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2fd77881-413c-4dc7-ba87-89551e0bf938 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:58.376664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2fd77881-413c-4dc7-ba87-89551e0bf938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:58.417270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:58.418188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.418586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.420454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.428698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:58.440589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:58.515285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 874/1761] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:58.516409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.516737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.517090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.569795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:58.571218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:58.584465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 875/1762] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:58.586912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:58.598619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc331b7-62de-4501-bf74-605b7ecc8589 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:58.606779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:58.607594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.608288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.608288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.609063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:24:58.609457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:58.611300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:58.615324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:58.623494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:58.632295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 876/1763] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:24:58.633307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:58.645469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b147d3-3138-4369-a4d4-99545c8af734 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:58.655023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:58.655885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:58.656351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:58.656828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:58.658279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 888/1764] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:24:58.658838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:58.662212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:58.725956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:58.735678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 877/1765] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:58.746527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6190c7-4cd4-4661-b1d8-372fecc163ed service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:59.061465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:24:59.096985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:59.098533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.100100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.100365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.101157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:59.113651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255683c0-e0f3-4c18-894a-21617d9e0aff admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:59.188340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 889/1766] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:24:58 2022] POST /identity/v3/users => generated 293 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:24:59.194702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:59.207258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533d182-3789-460f-9da6-e66714aed978 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:59.208367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.209634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.210565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.224292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:59.225518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.225518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.225891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.231393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:59.235782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:59.276266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 878/1767] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:59.284952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:59.310008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] MFA Rules not processed for user `644c0a532fa747c7abf4d53759e89181`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:59.312625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:59.317362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8701-a4bc-4f6c-a048-a63c29e4b062 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:59.343639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:59.344738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.348574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.348574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.350072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:59.354315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:59.417301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 879/1768] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:24:59.418383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:59.480768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a318f5-d8f9-4abf-889c-a018ecf7c640 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:24:59.505406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:24:59.506661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.506950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.507230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] PATH_INFO: `/v3/projects/ac34cea9ccf246b0b6a6cfda4ec17fb1/users/d4ad2ca62d0f4267b91177a05044548f/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.568945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac34cea9ccf246b0b6a6cfda4ec17fb1, user_id=d4ad2ca62d0f4267b91177a05044548f, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:24:59.572033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f16cf-ace2-43dd-b7df-bda2a0053d25 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:24:59.651111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 880/1769] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:24:59 2022] PUT /identity/v3/projects/ac34cea9ccf246b0b6a6cfda4ec17fb1/users/d4ad2ca62d0f4267b91177a05044548f/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:24:59.654696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:24:59.655245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:24:59.655740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:24:59.717850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 890/1770] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:59.727073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] MFA Rules not processed for user `d4ad2ca62d0f4267b91177a05044548f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:24:59.729308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:24:59.947299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 881/1771] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:24:59.950445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:24:59.983623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b35a10-12a7-4147-b829-31cea1edc05d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:00.339899 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:00.342156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:00.345837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:00.347356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:00.350296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:00.358228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:00.488057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 891/1772] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:24:59 2022] POST /identity/v3/projects => generated 385 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:00.490051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:00.528824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6d3f6-c372-4c9d-99a9-2d944406ecd5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:00.668364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:00.784930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:00.785991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:00.786473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:00.786921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:00.788230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:00.794686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:00.877426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 882/1773] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:00 2022] POST /identity/v3/users => generated 293 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:00.879301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:00.939883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbef6b-156a-48de-8b6c-b2e7680484af admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:00.960764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:00.960764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:00.960764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:00.960764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:00.962254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:00.971673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:01.020220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 892/1774] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:01.026744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:01.056133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b72e8-e922-4720-8efd-72e26a01291e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:01.066645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:01.067704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.068179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.069645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.071147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:01.073642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3f5e4-29a7-4963-986e-58e3c0bc0c97 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:01.096901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 883/1775] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:01.097896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:01.138225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7a3fb-688b-4151-a0b1-e76e6c4edd6b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:01.166651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:01.169069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.170229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.171093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] PATH_INFO: `/v3/projects/0ebd1d6d4689421cb4b02f241bc8f6da/users/63cf22b05a0d41888d7e5d52ce7e0cec/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.239163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebd1d6d4689421cb4b02f241bc8f6da, user_id=63cf22b05a0d41888d7e5d52ce7e0cec, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:01.251159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3201b-ef6e-46a3-8e50-6bb3b69ef124 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:01.330631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 893/1776] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:01 2022] PUT /identity/v3/projects/0ebd1d6d4689421cb4b02f241bc8f6da/users/63cf22b05a0d41888d7e5d52ce7e0cec/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:01.350574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc351-3999-425b-8063-0c1233b45105 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.351060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc351-3999-425b-8063-0c1233b45105 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.351535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc351-3999-425b-8063-0c1233b45105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.451110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-566cc351-3999-425b-8063-0c1233b45105 None None] MFA Rules not processed for user `63cf22b05a0d41888d7e5d52ce7e0cec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:01.453539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-566cc351-3999-425b-8063-0c1233b45105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:01.463306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96641-a2df-4d2e-942b-6d09302fbcca None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.463828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96641-a2df-4d2e-942b-6d09302fbcca None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.464628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96641-a2df-4d2e-942b-6d09302fbcca None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.467889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 894/1777] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 25 16:25:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:25:01.487123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.487960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.488399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.565071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:01.575230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:01.684415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:01.695862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 884/1778] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:01.704374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cc351-3999-425b-8063-0c1233b45105 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:01.729167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 895/1779] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:01.752910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:01.755610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:01.776810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07e063-1b26-4473-bf5c-2bbca5861195 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:01.790618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:01.791724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.792202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.792710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.830405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:01.833121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:01.852204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cc351-3999-425b-8063-0c1233b45105 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:01.863060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:01.864419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:01.865136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:01.866140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:01.868259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:01.871239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 896/1780] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:01.872648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:01.929908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 885/1781] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:01 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:01.934928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:01.976287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c9564-8c42-442a-961b-216ccba62c59 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:02.201636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:02.205150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:02.208259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:02.209707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:02.213240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:02.228431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:02.295975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 897/1782] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:01 2022] POST /identity/v3/users => generated 300 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:02.300190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:02.334803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4520d4-70d7-4f55-851e-2b3d817abd93 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:02.345647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:02.346952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:02.363171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:02.364759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:02.369282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:02.376467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:02.422448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:02.444567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 886/1783] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:02.455115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefd9d-9303-4778-be69-1bb66a26fee9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:02.470014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:02.471836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:02.473222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:02.477045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] PATH_INFO: `/v3/projects/2526db205c5b4874a47e5a18efe6a4ac/users/c77ac61652934f138323597609a2968f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:02.529226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2526db205c5b4874a47e5a18efe6a4ac, user_id=c77ac61652934f138323597609a2968f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:02.542648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:02.577678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 898/1784] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:02 2022] PUT /identity/v3/projects/2526db205c5b4874a47e5a18efe6a4ac/users/c77ac61652934f138323597609a2968f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:02.582291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:02.598600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce1cf6-2b2b-4d0b-a7d3-4d08fac3422a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:02.635554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:02.811932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:02.813030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:02.813225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:02.813957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:02.815258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:02.818594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:02.853041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 887/1785] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:02.858187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:02.890079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b80-7125-46c5-ba1b-afe6da360165 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:02.921940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:02.929585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:02.934010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:02.934721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] PATH_INFO: `/v3/projects/2526db205c5b4874a47e5a18efe6a4ac/users/c77ac61652934f138323597609a2968f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:02.942268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:02.945256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2526db205c5b4874a47e5a18efe6a4ac, user_id=c77ac61652934f138323597609a2968f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:02.946617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:02.950079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c16e3-ff35-4bf4-9210-762c891c9ff2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:03.001536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e34c-4b24-458f-84f3-1fbc81fb534e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:03.024600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 899/1786] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:02 2022] PUT /identity/v3/projects/2526db205c5b4874a47e5a18efe6a4ac/users/c77ac61652934f138323597609a2968f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:03.048790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:03.049401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:03.049912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:03.274872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] MFA Rules not processed for user `c77ac61652934f138323597609a2968f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:03.280995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:03.423194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:03.428014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:03.430888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:03.431797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] PATH_INFO: `/v3/users/bc7b85321d984136b2f6c0d771608460` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:03.450807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7b85321d984136b2f6c0d771608460)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:03.466601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:03.804175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7b85321d984136b2f6c0d771608460'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:03.846473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7b85321d984136b2f6c0d771608460'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:03.861454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 900/1787] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 814 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:03.894888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7b85321d984136b2f6c0d771608460'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:03.943964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:03.945258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7b85321d984136b2f6c0d771608460'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:03.946416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ec45503-4b09-437d-83e9-d5f0c7b5069a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7b85321d984136b2f6c0d771608460'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:03.949056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:03.980902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 888/1788] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:02 2022] DELETE /identity/v3/users/bc7b85321d984136b2f6c0d771608460 => generated 0 bytes in 1036 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:03.985154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:03.985587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:03.986259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:04.061788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:04.068222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:04.080826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:04.174418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:04.314802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085523f-f75a-45e5-88f7-21f2ae7a17e8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:04.325134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 889/1789] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:04.383476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:04.387065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:04.691255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:04.692766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:04.694657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:04.696886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:04.713129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:04.824314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73c3b4-052d-4d1d-88e1-4f08cdbd351e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:04.855390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:04.859947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:04.864186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:04.865383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:04.962070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:04.976548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:05.026761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 890/1790] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:05.075484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:05.086901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:05.147732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 901/1791] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 1205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:06.738307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:06.743174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:06.789496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa773f-e70b-4ec1-92b4-86343d073b45 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:06.799175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:06.800698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:06.801474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:06.802171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:06.805865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:06.952397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:06.953926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:07.028428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 891/1792] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:07.104103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:07.106549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:07.131509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90b1f-a37d-42bd-8903-12a555eb9c43 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:07.139938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:07.141084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:07.143459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:07.143920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] PATH_INFO: `/v3/projects/a539995e57a64c149016227046bd4446` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:07.148634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a539995e57a64c149016227046bd4446)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:07.153554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:07.331817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a539995e57a64c149016227046bd4446', 'request_id': 'req-396430a0-8e10-44e2-920e-e85d2f140954'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.365876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a539995e57a64c149016227046bd4446', 'request_id': 'req-396430a0-8e10-44e2-920e-e85d2f140954'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.391980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a539995e57a64c149016227046bd4446', 'request_id': 'req-396430a0-8e10-44e2-920e-e85d2f140954'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.417549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] The token cache is being invalidate because project a539995e57a64c149016227046bd4446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:07.417885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.419564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-396430a0-8e10-44e2-920e-e85d2f140954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.422391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 902/1793] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:07 2022] DELETE /identity/v3/projects/a539995e57a64c149016227046bd4446 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:07.428379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:07.456030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285380-a123-41aa-888e-e60b4c55e8aa service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:07.639730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:07.641408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:07.642186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:07.642697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] PATH_INFO: `/v3/users/1f289eee117d40ffb607d24cc6c565de` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:07.645255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f289eee117d40ffb607d24cc6c565de)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:07.649443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:07.679347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:07.756343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f289eee117d40ffb607d24cc6c565de'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.791737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f289eee117d40ffb607d24cc6c565de'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.808489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f289eee117d40ffb607d24cc6c565de'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.830256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f289eee117d40ffb607d24cc6c565de'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.831605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4992b7e-edc1-4db3-8f61-7cf6dc71d583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f289eee117d40ffb607d24cc6c565de'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:07.844234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 892/1794] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:07 2022] DELETE /identity/v3/users/1f289eee117d40ffb607d24cc6c565de => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:08.043992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:08.044905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:08.045819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:08.048861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 903/1795] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:25:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:25:08.057665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:08.057948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:08.058356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:08.087980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:08.089433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:08.228151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 893/1796] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:08.273697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:08.276385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:08.332974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:08.370517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a1081-3930-4b8c-8084-d8359b70c991 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:08.388038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:08.389632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:08.390144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:08.390640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:08.422629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:08.425547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16b52a-6140-4251-9c4d-b0828e444fba service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:08.450526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 904/1797] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:08.522969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:08.525600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:08.773578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadbe97-4d38-4e30-8af4-f56d884d94ca None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:08.784570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:08.786351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:08.786811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:08.790181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] PATH_INFO: `/v3/projects/e9765902d8c04c638916620be0c9231a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:08.793115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9765902d8c04c638916620be0c9231a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:08.797713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:08.873810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:08.932990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9765902d8c04c638916620be0c9231a', 'request_id': 'req-bb793c4a-b986-427d-9847-aa5caef0f605'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:08.951652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9765902d8c04c638916620be0c9231a', 'request_id': 'req-bb793c4a-b986-427d-9847-aa5caef0f605'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:08.981573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9765902d8c04c638916620be0c9231a', 'request_id': 'req-bb793c4a-b986-427d-9847-aa5caef0f605'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:09.015576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] The token cache is being invalidate because project e9765902d8c04c638916620be0c9231a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:09.015885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:09.018317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb793c4a-b986-427d-9847-aa5caef0f605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:09.042944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 894/1798] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:08 2022] DELETE /identity/v3/projects/e9765902d8c04c638916620be0c9231a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:09.044748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:09.045151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:09.045484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:09.092569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:09.095071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:09.290005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 905/1799] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:09.296537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:09.297127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:09.297720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:09.382271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:09.384459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:09.441456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 895/1800] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:09.441903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:09.470084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9a83-5994-4e93-b5a9-ee7ead757884 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:09.489445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:09.492380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:09.498089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:09.498832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:09.500729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:09.501206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:09.506283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:09.569373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 906/1801] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:09.570230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:09.619556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d84bc-12eb-4fcc-8dd9-5dd3c5339ad6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:09.636652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:09.640318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:09.641020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:09.642041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:09.643755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:09.660450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:09.842902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 896/1802] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:09 2022] POST /identity/v3/projects => generated 375 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:09.845375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:09.895757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284dddc-7cb9-42ab-8d33-59865f920d65 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:10.121530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:10.123559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:10.124170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:10.124777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:10.126310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:10.139355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:10.228807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:10.229663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 907/1803] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:09 2022] POST /identity/v3/users => generated 288 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:10.269703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1f27-58bb-4789-93fa-c7d78f49188a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:10.289449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:10.293053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:10.297151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:10.298014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:10.301553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:10.309765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:10.345953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 897/1804] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:10.351239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:10.378932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b18fc1-186d-46a7-9b03-dbe80799d207 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:10.409694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:10.409694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:10.409694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:10.409694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:10.410571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:10.413002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efa590-507b-465d-9c14-e397e971d650 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:10.470589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 908/1805] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:10.475701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:10.497672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea832-ff04-48e2-9ed3-3a45b28a41f6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:10.528419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:10.532167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:10.532763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:10.533241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] PATH_INFO: `/v3/projects/1fe652f363784ab3bba71ea704a24b94/users/8b36e0160e4a486988965eb3ff1f9eaa/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:10.580314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe652f363784ab3bba71ea704a24b94, user_id=8b36e0160e4a486988965eb3ff1f9eaa, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:10.585196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:10.621401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 898/1806] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:10 2022] PUT /identity/v3/projects/1fe652f363784ab3bba71ea704a24b94/users/8b36e0160e4a486988965eb3ff1f9eaa/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:10.633980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:10.635153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:10.635621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:10.763608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] MFA Rules not processed for user `8b36e0160e4a486988965eb3ff1f9eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:10.765894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:11.006747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:11.153637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 909/1807] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:11.161904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:11.471403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96a0a-5836-41d2-9b6e-5c41c31dc8de admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:11.492470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:11.493236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:11.494978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:11.495305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:11.496183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:11.502931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:11.606746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 899/1808] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:11 2022] POST /identity/v3/projects => generated 375 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:11.611398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:11.633315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8666b73-e898-48c8-99ce-5b034cbfea7f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:11.749101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:11.750221 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:11.750535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:11.751003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:11.752389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:11.757656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:11.817188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 910/1809] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:11 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:11.823618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:11.849955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d35caf-1aac-40f4-af77-a2c0df3b1543 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:11.867927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:11.869129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:11.869485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:11.870350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:11.871132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:11.873003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:11.937129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 900/1810] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:11.943531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:11.967485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d959626-ef7c-49c7-9303-16aa0e1b2715 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:11.980304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:11.981648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:11.981790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:11.982616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] PATH_INFO: `/v3/projects/a6ed080a9be74314800fc54d575464bb/users/200698f0d3004db4a22cea4cbb52754d/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:12.077406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed080a9be74314800fc54d575464bb, user_id=200698f0d3004db4a22cea4cbb52754d, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:12.082813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:12.107072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 911/1811] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:11 2022] PUT /identity/v3/projects/a6ed080a9be74314800fc54d575464bb/users/200698f0d3004db4a22cea4cbb52754d/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:12.113536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:12.139614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc36241-15d5-4668-8842-b38d136d5740 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:12.182048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:12.265061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:12.267102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:12.267627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:12.268120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:12.269590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:12.272938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328e9c1-bca3-4c03-9c46-aa4ce9017ba0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:12.297448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 901/1812] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:12.303363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:12.322339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de50f-6c66-4a03-ba21-3c610d0fa038 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:12.330555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:12.331773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:12.332095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:12.332443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] PATH_INFO: `/v3/projects/a6ed080a9be74314800fc54d575464bb/users/200698f0d3004db4a22cea4cbb52754d/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:12.334956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed080a9be74314800fc54d575464bb, user_id=200698f0d3004db4a22cea4cbb52754d, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:12.337858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:12.350444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 912/1813] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:12 2022] PUT /identity/v3/projects/a6ed080a9be74314800fc54d575464bb/users/200698f0d3004db4a22cea4cbb52754d/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:12.367715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:12.368132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:12.368834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:12.425815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] MFA Rules not processed for user `200698f0d3004db4a22cea4cbb52754d`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:12.428050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:12.532593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 902/1814] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:12.546810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:12.549040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:12.650662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857722-f3ed-4a61-a4b6-ce8f46f6dd0c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:12.657510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:12.658682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:12.659157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:12.659651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:12.764553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:12.788380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:12.790700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb856-86ef-423b-acb3-2e1c2ca75202 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:12.803587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 913/1815] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:16.412448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:16.428502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a92143-696c-4a1e-a7c2-1574b0280b22 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:16.438365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:16.440122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:16.442026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:16.443197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:16.479775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:16.668997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:16.670566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98ec96-3c4e-4963-b0b5-0385f9ef7c6e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:16.699856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 903/1816] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:16.954204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:16.955665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:16.956444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:16.960620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 914/1817] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:25:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:25:16.997916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:16.998974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:17.000461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:17.071111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:17.089447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:17.256005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:17.371949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 904/1818] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:25:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:17.416820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:17.420863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:17.552466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f2ecd-56bb-45d4-9455-8c5e48bc9d16 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:17.604594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:17.607126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:17.608005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:17.611898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:17.657105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:17.661439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4de094-5ad4-4c7c-9816-35fd6ff71809 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:17.706016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 915/1819] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:17.749327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:17.755801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:17.985217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c29a8-3eec-40a5-a13a-3c1dcb6ad1b6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:17.999259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:18.001883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:18.002800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:18.004218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] PATH_INFO: `/v3/users/961d468934304d73aa7865dcb53a20c1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:18.009165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961d468934304d73aa7865dcb53a20c1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:18.019193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:18.183607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961d468934304d73aa7865dcb53a20c1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:18.199289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961d468934304d73aa7865dcb53a20c1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:18.211950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961d468934304d73aa7865dcb53a20c1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:18.241621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961d468934304d73aa7865dcb53a20c1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:18.242705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961d468934304d73aa7865dcb53a20c1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:18.254350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 905/1820] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:17 2022] DELETE /identity/v3/users/961d468934304d73aa7865dcb53a20c1 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:18.262105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:18.262105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:18.263132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:18.304229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:18.305760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:18.458942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 916/1821] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:18.471921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:18.474675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:18.626342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaa94d-f417-46f7-b2bc-6cec4fb93195 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:18.633132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:18.634609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:18.635285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:18.635779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:18.670357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:18.674304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:18.682785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:18.695063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 906/1822] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:19.213052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:19.215454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:19.245847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e7a4-15f6-4f30-a842-2c14ce77526d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:19.262276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:19.263269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:19.263696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:19.263898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] PATH_INFO: `/v3/projects/c28e22852a274c6fa26da14da20f47e4` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:19.265795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28e22852a274c6fa26da14da20f47e4)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:19.268946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:19.363348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28e22852a274c6fa26da14da20f47e4', 'request_id': 'req-22048525-8859-4f15-bbe7-5c115c3c3dfc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.371121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28e22852a274c6fa26da14da20f47e4', 'request_id': 'req-22048525-8859-4f15-bbe7-5c115c3c3dfc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.382310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28e22852a274c6fa26da14da20f47e4', 'request_id': 'req-22048525-8859-4f15-bbe7-5c115c3c3dfc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.403403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] The token cache is being invalidate because project c28e22852a274c6fa26da14da20f47e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:19.403801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.405054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.407346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 917/1823] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:19 2022] DELETE /identity/v3/projects/c28e22852a274c6fa26da14da20f47e4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:19.413818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:19.427407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a63b08-33ac-4338-a98c-7407e236a479 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:19.551134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:19.552697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:19.553202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:19.553734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] PATH_INFO: `/v3/users/33a480f2ed4d4fefba1f65cdf1fcaec3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:19.556240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a480f2ed4d4fefba1f65cdf1fcaec3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:19.560086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:19.640496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a480f2ed4d4fefba1f65cdf1fcaec3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.656805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a480f2ed4d4fefba1f65cdf1fcaec3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.671252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a480f2ed4d4fefba1f65cdf1fcaec3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.691702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a480f2ed4d4fefba1f65cdf1fcaec3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.692631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a480f2ed4d4fefba1f65cdf1fcaec3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:19.699484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 907/1824] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:19 2022] DELETE /identity/v3/users/33a480f2ed4d4fefba1f65cdf1fcaec3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:20.179317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:20.182871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:20.215224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22048525-8859-4f15-bbe7-5c115c3c3dfc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:20.370619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:20.372086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:20.374433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.375186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.375910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] PATH_INFO: `/v3/projects/d110540babfd41c69bbabcb0673d5463` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.378548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d110540babfd41c69bbabcb0673d5463)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:20.379511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:20.383985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:20.391875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:20.470182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd110540babfd41c69bbabcb0673d5463', 'request_id': 'req-9bd95289-982e-4155-a1c0-5b0f6d2e5221'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:20.479143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd110540babfd41c69bbabcb0673d5463', 'request_id': 'req-9bd95289-982e-4155-a1c0-5b0f6d2e5221'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:20.487670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd110540babfd41c69bbabcb0673d5463', 'request_id': 'req-9bd95289-982e-4155-a1c0-5b0f6d2e5221'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:20.495604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0848-1925-481a-a0bd-5f684f562c98 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:20.508550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] The token cache is being invalidate because project d110540babfd41c69bbabcb0673d5463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:20.509168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:20.510531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9bd95289-982e-4155-a1c0-5b0f6d2e5221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:20.512758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 918/1825] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:20 2022] DELETE /identity/v3/projects/d110540babfd41c69bbabcb0673d5463 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:20.521157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349846f6-7a9c-4d52-a58f-41a2281f0bdf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.521643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349846f6-7a9c-4d52-a58f-41a2281f0bdf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.522033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349846f6-7a9c-4d52-a58f-41a2281f0bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.554010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:20.566789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-349846f6-7a9c-4d52-a58f-41a2281f0bdf None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:20.569139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-349846f6-7a9c-4d52-a58f-41a2281f0bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:20.593080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:20.595384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.595981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.596516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.600175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:20.692558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 919/1826] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:20.707059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.707575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.708083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.735390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:20.738277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:20.740999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:20.743345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:20.759855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 920/1827] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:20.760718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 908/1828] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:20.770686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:20.784416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd499-6b15-4a94-b31a-a93a402e192d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:20.795386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:20.797776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.797991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.798305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.799546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:20.799868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:20.803440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:20.825411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 921/1829] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:20.829929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:20.859603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809395db-f7cc-4785-874d-c49e412cf87c service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:20.867671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:20.869599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:20.870174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:20.870800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:20.873384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:20.876225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:20.921657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 909/1830] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:20 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:20.926441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:20.947035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6569877-8979-42a7-9590-171ed869d899 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.064438 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.065906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.066381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.067449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.068145 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.071344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.113403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 922/1831] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:20 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:21.121453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.142932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7d29-0929-4a47-b85c-b1ad36d36f0f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.149920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.150862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.151173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.151474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.152279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.154384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.179375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 910/1832] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:21.184567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.192891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:21.201712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc2179-9150-434e-8ad0-b9dabc2678ee admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.216290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.217588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.218128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.227596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.227596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.227596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d45108-5dab-44f3-af27-bcc648554cb7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.248533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 923/1833] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:21.257069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.278360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec707918-be80-43c7-a4bc-4731fcb576c4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.289418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.290421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.290734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.291018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] PATH_INFO: `/v3/projects/4f767595ef974be8a65fe024af34f11d/users/963da227fab04994b923e58a556e850d/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.313037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f767595ef974be8a65fe024af34f11d, user_id=963da227fab04994b923e58a556e850d, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.318708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.345646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 911/1834] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:21 2022] PUT /identity/v3/projects/4f767595ef974be8a65fe024af34f11d/users/963da227fab04994b923e58a556e850d/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:21.363211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225540-4635-4338-bb67-fee14167da89 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.363637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225540-4635-4338-bb67-fee14167da89 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.364189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11225540-4635-4338-bb67-fee14167da89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.430677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-11225540-4635-4338-bb67-fee14167da89 None None] MFA Rules not processed for user `963da227fab04994b923e58a556e850d`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:21.432537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11225540-4635-4338-bb67-fee14167da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:21.576407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 924/1835] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:21.582648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.678648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d86348-a1a2-4c91-a615-dd9d269058a6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.686164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.686876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.687185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.687467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.688224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.690428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.733233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 912/1836] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:21 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:21.739431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11225540-4635-4338-bb67-fee14167da89 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.754437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11225540-4635-4338-bb67-fee14167da89 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.849473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.850090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.850468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.850750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.851472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.854253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.886572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 925/1837] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:21 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:21.892571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.911467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31465-56b8-4dfb-b897-c4fadf5fcd7d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.919272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.920059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.920365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.920660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.921558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:21.923309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:21.930704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:21.939050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 913/1838] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:21.945724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:21.961077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6966-fabd-4378-bb97-b69bb3c327ad admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:21.970128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:21.971677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:21.972192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:21.973629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] PATH_INFO: `/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65/users/06fb776873b84b1ea7f127c1b418a34e/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:21.998656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca55b8adccc4b9f8e08d3277142ae65, user_id=06fb776873b84b1ea7f127c1b418a34e, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:22.002045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:22.039828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 926/1839] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:21 2022] PUT /identity/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65/users/06fb776873b84b1ea7f127c1b418a34e/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:22.044938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:22.057092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed01ef-6761-401b-b73f-b3b227282665 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:22.178677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:22.180708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:22.181320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:22.181959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:22.183905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:22.187051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b6146-f901-4f5e-bd78-8a684ee76e97 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:22.216586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 914/1840] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:22.223494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:22.243563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf7598-4606-40b9-9984-652c671a0d3b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:22.253079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:22.254921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:22.255411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:22.255912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] PATH_INFO: `/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65/users/06fb776873b84b1ea7f127c1b418a34e/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:22.260136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca55b8adccc4b9f8e08d3277142ae65, user_id=06fb776873b84b1ea7f127c1b418a34e, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:22.265812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:22.288727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 927/1841] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:22 2022] PUT /identity/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65/users/06fb776873b84b1ea7f127c1b418a34e/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:22.300420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:22.300761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:22.301087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:22.336363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] MFA Rules not processed for user `06fb776873b84b1ea7f127c1b418a34e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:22.337864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:22.461063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 915/1842] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:22.477551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:22.483662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:22.538652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:22.604228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6482c99-c1a2-4aef-9739-aa717bf4827c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:22.610262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:22.611716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:22.612234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:22.612743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:22.634976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:22.638278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af601a-b148-4042-bd1b-74bb7b83ed59 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:22.657232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 928/1843] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:23.420373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:23.547700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cc88-58a1-437a-8ebf-2e541db5a62e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:23.555830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:23.557663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:23.558155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:23.558664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] PATH_INFO: `/v3/users/963da227fab04994b923e58a556e850d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:23.561307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963da227fab04994b923e58a556e850d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:23.565551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:23.616448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:23.669220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963da227fab04994b923e58a556e850d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:23.683691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963da227fab04994b923e58a556e850d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:23.695993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963da227fab04994b923e58a556e850d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:23.726088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963da227fab04994b923e58a556e850d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:23.727142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963da227fab04994b923e58a556e850d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:23.739083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 916/1844] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:23 2022] DELETE /identity/v3/users/963da227fab04994b923e58a556e850d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:23.745650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:23.746259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:23.746402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:23.772228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:23.774228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:23.873200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 929/1845] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:23.886020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:23.887804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:23.973290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b08f6-4f76-4ce7-9378-eb991f209222 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:23.978926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:23.979889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:23.980268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:23.980722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:23.994338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:23.996890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:24.012402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 917/1846] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:24.454337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:24.457307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:24.486125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3b0bc-a08d-4c45-a14b-c156c2d0edde None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:24.497762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:24.500445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:24.501051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:24.501691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] PATH_INFO: `/v3/projects/4f767595ef974be8a65fe024af34f11d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:24.504244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f767595ef974be8a65fe024af34f11d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:24.523576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:24.625520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f767595ef974be8a65fe024af34f11d', 'request_id': 'req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.635182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f767595ef974be8a65fe024af34f11d', 'request_id': 'req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.645222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f767595ef974be8a65fe024af34f11d', 'request_id': 'req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.665672 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] The token cache is being invalidate because project 4f767595ef974be8a65fe024af34f11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:24.665972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.666655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.668406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 930/1847] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:24 2022] DELETE /identity/v3/projects/4f767595ef974be8a65fe024af34f11d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:24.673217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:24.689412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340dd11-86cd-4b5b-b235-0e006e4210d0 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:24.794961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:24.796752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:24.797300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:24.797918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] PATH_INFO: `/v3/users/06fb776873b84b1ea7f127c1b418a34e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:24.799842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fb776873b84b1ea7f127c1b418a34e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:24.802494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:24.884321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:24.887634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fb776873b84b1ea7f127c1b418a34e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.902206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fb776873b84b1ea7f127c1b418a34e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.919948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fb776873b84b1ea7f127c1b418a34e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.951141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fb776873b84b1ea7f127c1b418a34e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.952377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-56e8e442-5c0f-469f-82b4-a60c523d9f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fb776873b84b1ea7f127c1b418a34e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:24.963885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 918/1848] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:24 2022] DELETE /identity/v3/users/06fb776873b84b1ea7f127c1b418a34e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:26.336639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:26.338888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:26.403236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ff2e-1aba-45d3-a7ae-a1f9d1f7abdf admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:26.796977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:26.809382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:26.809382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:26.809382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] PATH_INFO: `/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:26.818629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca55b8adccc4b9f8e08d3277142ae65)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:26.825844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:26.834393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:27.043565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca55b8adccc4b9f8e08d3277142ae65', 'request_id': 'req-c3277454-cf00-434a-83d7-72fb29fe3665'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.072315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca55b8adccc4b9f8e08d3277142ae65', 'request_id': 'req-c3277454-cf00-434a-83d7-72fb29fe3665'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.112817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca55b8adccc4b9f8e08d3277142ae65', 'request_id': 'req-c3277454-cf00-434a-83d7-72fb29fe3665'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.204604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] The token cache is being invalidate because project 7ca55b8adccc4b9f8e08d3277142ae65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:27.207292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.212670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.222653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 931/1849] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:26 2022] DELETE /identity/v3/projects/7ca55b8adccc4b9f8e08d3277142ae65 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:27.230542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d22c4-0bb2-4d78-bcab-d287c804ab2e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:27.231303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d22c4-0bb2-4d78-bcab-d287c804ab2e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:27.231604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d22c4-0bb2-4d78-bcab-d287c804ab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:27.313145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-da4d22c4-0bb2-4d78-bcab-d287c804ab2e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:27.320590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da4d22c4-0bb2-4d78-bcab-d287c804ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:27.369033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:27.372752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:27.606284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 919/1850] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:27.619881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:27.620863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:27.621686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:27.625014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3277454-cf00-434a-83d7-72fb29fe3665 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:27.635730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:27.638742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:27.639266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:27.640002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] PATH_INFO: `/v3/users/d4ad2ca62d0f4267b91177a05044548f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:27.642951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ad2ca62d0f4267b91177a05044548f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:27.645621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:27.716007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:27.719363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:27.748179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 920/1851] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:27.750952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:27.779849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b5d80-c1af-4eb2-8914-4cb2409b3e70 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:27.787909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ad2ca62d0f4267b91177a05044548f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.789718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:27.791189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:27.791701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:27.794749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:27.798995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:27.799568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:27.804111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:27.829417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ad2ca62d0f4267b91177a05044548f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.832409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ad2ca62d0f4267b91177a05044548f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.855272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ad2ca62d0f4267b91177a05044548f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.857040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-10aa4c65-e967-4c87-986e-865bfd50af78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ad2ca62d0f4267b91177a05044548f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:27.866868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 921/1852] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:27.875225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:27.881125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:27.947169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cbf5-9e0a-4647-a24f-c775443ba1c9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:27.957464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 932/1853] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:27 2022] DELETE /identity/v3/users/d4ad2ca62d0f4267b91177a05044548f => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:27.962745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:27.965246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:27.965871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:28.031973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:28.034937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:28.043609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:28.146113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:28.152644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:28.155395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:28.161137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:28.162638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:28.171370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:28.214415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 933/1854] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:28.286563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:28.290603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:28.319411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 922/1855] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:27 2022] POST /identity/v3/projects => generated 385 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:28.326029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:28.442192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389f86-0c10-4d3e-bdd4-dfabaa18ca50 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:28.483223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:28.607945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:28.609632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:28.610246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:28.611952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:28.728862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991310-9e71-41cd-bf91-ba3c5512fb9a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:28.747609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:28.750752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:28.751288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:28.769914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:28.780894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:28.784901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:28.880047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 923/1856] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:28 2022] POST /identity/v3/users => generated 293 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:28.883311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:28.884005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:28.888316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:28.940732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 934/1857] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:28.950314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897e729-f054-4ff8-8c85-d1777b7e12c5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:28.963285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:28.965765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:28.966851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:28.967234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:28.968894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:28.975100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:29.015828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 924/1858] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:29.023186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.047348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85a61-e50d-41fc-a6e7-f28a3057e009 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.064704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:29.069690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.072081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.074423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.079086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:29.088973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a3511-f84d-47c0-ac7c-f6978d54c3fd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:29.134928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 935/1859] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:29.138449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.175398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db188e-0e7c-4c76-b2be-63f795da4562 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.184605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:29.186224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.186820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.187614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] PATH_INFO: `/v3/projects/8635ce72fdaf46dfa8a92b3c3d8c2c71/users/6ae58c9f1efa43c7b8c29989fe086dd1/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.244805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8635ce72fdaf46dfa8a92b3c3d8c2c71, user_id=6ae58c9f1efa43c7b8c29989fe086dd1, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:29.253137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:29.292024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 925/1860] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:29 2022] PUT /identity/v3/projects/8635ce72fdaf46dfa8a92b3c3d8c2c71/users/6ae58c9f1efa43c7b8c29989fe086dd1/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:29.311828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.312318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.312810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.313924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.316965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:29.328001 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:29.368744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] MFA Rules not processed for user `6ae58c9f1efa43c7b8c29989fe086dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:29.371648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:29.462116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288d13e-0bfc-4e38-b9e2-2897954d764c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.470189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:29.470731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.471132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.471473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] PATH_INFO: `/v3/users/590940ccd5374abea292d1e4d6c94e87` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.473619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590940ccd5374abea292d1e4d6c94e87)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:29.476911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:29.530192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 936/1861] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:29.532357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.568583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2c97-1ff4-44fd-847d-245708fe45dd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.580845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:29.583777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.583777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.583777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.585289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:29.589139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590940ccd5374abea292d1e4d6c94e87'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:29.589942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:29.604678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590940ccd5374abea292d1e4d6c94e87'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:29.621203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590940ccd5374abea292d1e4d6c94e87'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:29.654904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590940ccd5374abea292d1e4d6c94e87'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:29.656052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3f2fb5bd-c01e-45b4-aaad-512d5732d271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590940ccd5374abea292d1e4d6c94e87'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:29.666654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 937/1862] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:29 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:29.668592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.676925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 926/1863] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:29 2022] DELETE /identity/v3/users/590940ccd5374abea292d1e4d6c94e87 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:29.677900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:29.684944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.685692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.686742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.705654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:29.737134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:29.739564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:29.831136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33646a-1c2e-4231-9349-c9bea9fe45ac admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.942832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 927/1864] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:29.946764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:29.950224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:29.971565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaf212-df39-4dd9-bd35-563f659125f6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:29.993583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:29.995217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:29.995986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:29.996519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] PATH_INFO: `/v3/projects/ac34cea9ccf246b0b6a6cfda4ec17fb1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:29.998976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac34cea9ccf246b0b6a6cfda4ec17fb1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.002416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.021563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.023847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.024284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.024706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.025851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.028428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.064788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 938/1865] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:29 2022] POST /identity/v3/users => generated 300 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:30.065997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.068457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:30.087479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:30.108854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac34cea9ccf246b0b6a6cfda4ec17fb1', 'request_id': 'req-503595f2-d990-4277-8a86-2bc5cbf2e43c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.125124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac34cea9ccf246b0b6a6cfda4ec17fb1', 'request_id': 'req-503595f2-d990-4277-8a86-2bc5cbf2e43c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.141195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac34cea9ccf246b0b6a6cfda4ec17fb1', 'request_id': 'req-503595f2-d990-4277-8a86-2bc5cbf2e43c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.167514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] The token cache is being invalidate because project ac34cea9ccf246b0b6a6cfda4ec17fb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:30.167855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.169052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.172333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 928/1866] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:29 2022] DELETE /identity/v3/projects/ac34cea9ccf246b0b6a6cfda4ec17fb1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:30.179010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.181382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:30.211039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c9a99-1416-4ef1-abd6-f6aa52a160ea admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.322682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.326120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503595f2-d990-4277-8a86-2bc5cbf2e43c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.329409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.329409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.329409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.333207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:30.337955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.340447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.341156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.342254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.343720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.347727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.359295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.362767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.368277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 929/1867] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:30.369969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.374679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:30.387625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 939/1868] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:30.393890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.400762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eaddd-1724-4203-8163-3a064eb7f095 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.411236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.413687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.414520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.415029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f541be-5c41-4657-9d33-0e9ecad5b9e5 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.415566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] PATH_INFO: `/v3/users/63cf22b05a0d41888d7e5d52ce7e0cec` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.418080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] RBAC: Authorizing `identity:delete_user(user_id=63cf22b05a0d41888d7e5d52ce7e0cec)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.423573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.424971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.428397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.429164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.430368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] PATH_INFO: `/v3/projects/d5fd7ab2f160465bb87364c513e48f29/users/6a048da3552141aa941b7aa2ed191928/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.460417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:30.482059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fd7ab2f160465bb87364c513e48f29, user_id=6a048da3552141aa941b7aa2ed191928, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.495554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.561938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 940/1869] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:30 2022] PUT /identity/v3/projects/d5fd7ab2f160465bb87364c513e48f29/users/6a048da3552141aa941b7aa2ed191928/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:30.563469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.572661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63cf22b05a0d41888d7e5d52ce7e0cec'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.587333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63cf22b05a0d41888d7e5d52ce7e0cec'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.600537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63cf22b05a0d41888d7e5d52ce7e0cec'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.629059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63cf22b05a0d41888d7e5d52ce7e0cec'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.631865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63cf22b05a0d41888d7e5d52ce7e0cec'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:30.655029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 930/1870] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:30 2022] DELETE /identity/v3/users/63cf22b05a0d41888d7e5d52ce7e0cec => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:30.736954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640e06-def4-44b5-a5bd-504ceb5f6b4c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.871546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:30.899260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.901261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.901730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.903299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.905097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:30.909834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315e4db-5125-45cf-8eac-112d773964e2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:30.927531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 941/1871] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:30.933317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:30.935341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:30.966625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a79bbf-5c9b-4de5-9b2d-834c217750da admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:30.989771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:30.991817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:30.992249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:30.993013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] PATH_INFO: `/v3/projects/d5fd7ab2f160465bb87364c513e48f29/users/6a048da3552141aa941b7aa2ed191928/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:30.998005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fd7ab2f160465bb87364c513e48f29, user_id=6a048da3552141aa941b7aa2ed191928, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.004037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:31.036029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 931/1872] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:30 2022] PUT /identity/v3/projects/d5fd7ab2f160465bb87364c513e48f29/users/6a048da3552141aa941b7aa2ed191928/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:31.052600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.054157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.054613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.142822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] MFA Rules not processed for user `6a048da3552141aa941b7aa2ed191928`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:31.144807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.288718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:31.292487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.297573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 942/1873] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:31.322351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:31.325094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.442943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2930b98-7169-4db0-9433-76b7ee8a2764 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:31.449522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:31.451102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.451962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.452577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.483094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.486584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:31.493553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2afdaa-8b89-4c71-8382-abb3eacd0e38 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:31.508686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 943/1874] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:31.510705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:31.517772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.520458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:31.521540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.521908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.522152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] PATH_INFO: `/v3/projects/0b1c8c7afd664267ae53664f5747a98d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.538346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1c8c7afd664267ae53664f5747a98d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.558300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645833a4-ab5d-4da6-b7d3-09f2276f623e service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:31.565539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:31.566568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.566861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.567183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] PATH_INFO: `/v3/projects/0ebd1d6d4689421cb4b02f241bc8f6da` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.567858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:31.569074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebd1d6d4689421cb4b02f241bc8f6da)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.572173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:31.663057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebd1d6d4689421cb4b02f241bc8f6da', 'request_id': 'req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.692512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebd1d6d4689421cb4b02f241bc8f6da', 'request_id': 'req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.702065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebd1d6d4689421cb4b02f241bc8f6da', 'request_id': 'req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.717322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1c8c7afd664267ae53664f5747a98d', 'request_id': 'req-0838351e-6929-4f47-b52f-90851cb652d0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.718244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:31.730209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1c8c7afd664267ae53664f5747a98d', 'request_id': 'req-0838351e-6929-4f47-b52f-90851cb652d0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.731197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] The token cache is being invalidate because project 0ebd1d6d4689421cb4b02f241bc8f6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:31.732840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.734643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.738149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 944/1875] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:31 2022] DELETE /identity/v3/projects/0ebd1d6d4689421cb4b02f241bc8f6da => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:31.745236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:31.747312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1c8c7afd664267ae53664f5747a98d', 'request_id': 'req-0838351e-6929-4f47-b52f-90851cb652d0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.749074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.773512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] The token cache is being invalidate because project 0b1c8c7afd664267ae53664f5747a98d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:31.774022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.775131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:31.782360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 932/1876] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:31 2022] DELETE /identity/v3/projects/0b1c8c7afd664267ae53664f5747a98d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:31.782360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:31.782360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:31.806919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:31.922403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838351e-6929-4f47-b52f-90851cb652d0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:31.926111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70271f-eb3f-4e13-89d4-cb3feb1d03f2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:31.929760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:31.930977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.931490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.931916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] PATH_INFO: `/v3/users/c77ac61652934f138323597609a2968f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.933714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77ac61652934f138323597609a2968f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.934583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:31.936235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:31.936429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:31.936740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:31.937238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] PATH_INFO: `/v3/users/6ae58c9f1efa43c7b8c29989fe086dd1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:31.939625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae58c9f1efa43c7b8c29989fe086dd1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:31.943964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:32.035489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77ac61652934f138323597609a2968f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.043858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae58c9f1efa43c7b8c29989fe086dd1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.055848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77ac61652934f138323597609a2968f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.058782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae58c9f1efa43c7b8c29989fe086dd1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.059856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae58c9f1efa43c7b8c29989fe086dd1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.072159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77ac61652934f138323597609a2968f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.077059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae58c9f1efa43c7b8c29989fe086dd1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.079070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2802210-1e42-4c98-b164-67b23566c6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae58c9f1efa43c7b8c29989fe086dd1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.103614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77ac61652934f138323597609a2968f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.105010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77ac61652934f138323597609a2968f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.129123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 933/1877] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:31 2022] DELETE /identity/v3/users/c77ac61652934f138323597609a2968f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:32.129123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:32.144891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:32.156224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 945/1878] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:31 2022] DELETE /identity/v3/users/6ae58c9f1efa43c7b8c29989fe086dd1 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:32.164402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:32.164700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:32.165173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:32.234541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:32.242369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:32.445015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1ece3-f473-4c27-a821-6c7f1298b2bb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:32.533032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 946/1879] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:32.543822 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:32.582225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:32.585518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:32.685529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:32.689170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:32.691671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:32.692399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] PATH_INFO: `/v3/users/dd90c7bcd8c646acaf371acee1ad0f79` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:32.700993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd90c7bcd8c646acaf371acee1ad0f79)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:32.745073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:32.756710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:32.967024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9de36-0c5c-4493-90cd-a64c70155f36 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:32.978039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:32.979383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:32.980005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:32.980534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd90c7bcd8c646acaf371acee1ad0f79'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:32.980741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:33.001602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:33.005162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:33.037422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd90c7bcd8c646acaf371acee1ad0f79'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:33.037695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 947/1880] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:33.061992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd90c7bcd8c646acaf371acee1ad0f79'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:33.120077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd90c7bcd8c646acaf371acee1ad0f79'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:33.123538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd90c7bcd8c646acaf371acee1ad0f79'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:33.148310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 934/1881] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:32 2022] DELETE /identity/v3/users/dd90c7bcd8c646acaf371acee1ad0f79 => generated 0 bytes in 1025 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:33.576632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:33.579086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:33.622550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013881f9-1a4b-488c-84d7-9c1f15e1e161 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:33.853020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:33.855116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:33.855517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:33.856137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] PATH_INFO: `/v3/projects/2526db205c5b4874a47e5a18efe6a4ac` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:33.858186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2526db205c5b4874a47e5a18efe6a4ac)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:33.864804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:33.930629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:33.934092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:33.963169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c514af-a5ae-43ae-ab2f-414b1ae2f903 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:33.992732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2526db205c5b4874a47e5a18efe6a4ac', 'request_id': 'req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.008297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2526db205c5b4874a47e5a18efe6a4ac', 'request_id': 'req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.023835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2526db205c5b4874a47e5a18efe6a4ac', 'request_id': 'req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.055886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] The token cache is being invalidate because project 2526db205c5b4874a47e5a18efe6a4ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:34.056396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.058705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa3c4e3a-c869-4a61-b66d-29dc45af43dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.062086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 948/1882] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:33 2022] DELETE /identity/v3/projects/2526db205c5b4874a47e5a18efe6a4ac => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:34.074889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91d2-b63f-446d-9e3f-df69d41bf306 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.075392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91d2-b63f-446d-9e3f-df69d41bf306 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.076088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91d2-b63f-446d-9e3f-df69d41bf306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.081315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.084114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.085099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.085922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] PATH_INFO: `/v3/users/b0bfbe80c5b84bbdac6af9e6595a8bc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.088766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0bfbe80c5b84bbdac6af9e6595a8bc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:34.093549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:34.126627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-659f91d2-b63f-446d-9e3f-df69d41bf306 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:34.128928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-659f91d2-b63f-446d-9e3f-df69d41bf306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.155099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 949/1883] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:34.170952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.170952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.171456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.212082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:34.214873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.236661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0bfbe80c5b84bbdac6af9e6595a8bc9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.237792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 950/1884] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:34.239675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.260423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aefe7-96b3-4163-bf79-574c0cce5c9b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:34.263190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0bfbe80c5b84bbdac6af9e6595a8bc9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.269994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:34.278068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.279927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.280712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.281308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0bfbe80c5b84bbdac6af9e6595a8bc9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.281523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.283140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:34.283697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:34.287680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97977b0e-389e-4142-82cf-e07f0deccb76 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:34.310531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0bfbe80c5b84bbdac6af9e6595a8bc9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.312024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0bfbe80c5b84bbdac6af9e6595a8bc9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.323926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 935/1885] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:33 2022] DELETE /identity/v3/users/b0bfbe80c5b84bbdac6af9e6595a8bc9 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:34.325651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.328108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.335521 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 951/1886] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:34.340454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.340980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.341603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.386743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:34.389529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.449966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:34.487779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b664ee3-900f-4f0c-bdf7-4b57b4e0ba2a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:34.496205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.498141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.498702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.499253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] PATH_INFO: `/v3/projects/8635ce72fdaf46dfa8a92b3c3d8c2c71` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.501543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8635ce72fdaf46dfa8a92b3c3d8c2c71)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:34.505419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:34.523426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 952/1887] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:34.524498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.526357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.549721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02068a70-5965-4c83-b4d5-43bf4c2478bc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:34.562002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.563647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.564351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.564970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] PATH_INFO: `/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.568683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee26e5c53e5b40de92f8cee9a40ddc12)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:34.574350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:34.624098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8635ce72fdaf46dfa8a92b3c3d8c2c71', 'request_id': 'req-516f610a-ac21-41d6-873b-57d4be414f41'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.648016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8635ce72fdaf46dfa8a92b3c3d8c2c71', 'request_id': 'req-516f610a-ac21-41d6-873b-57d4be414f41'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.664526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8635ce72fdaf46dfa8a92b3c3d8c2c71', 'request_id': 'req-516f610a-ac21-41d6-873b-57d4be414f41'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.679908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] The token cache is being invalidate because project 8635ce72fdaf46dfa8a92b3c3d8c2c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:34.680360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.681261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.683760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 936/1888] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:34 2022] DELETE /identity/v3/projects/8635ce72fdaf46dfa8a92b3c3d8c2c71 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:34.685124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.687184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.701733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee26e5c53e5b40de92f8cee9a40ddc12', 'request_id': 'req-dcff3f90-85cf-4937-8e9c-4e525d081590'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.709649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee26e5c53e5b40de92f8cee9a40ddc12', 'request_id': 'req-dcff3f90-85cf-4937-8e9c-4e525d081590'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.720263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee26e5c53e5b40de92f8cee9a40ddc12', 'request_id': 'req-dcff3f90-85cf-4937-8e9c-4e525d081590'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.740997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] The token cache is being invalidate because project ee26e5c53e5b40de92f8cee9a40ddc12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:34.741458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.742372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:34.745008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 953/1889] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:34 2022] DELETE /identity/v3/projects/ee26e5c53e5b40de92f8cee9a40ddc12 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:34.746440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.748533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.797049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f610a-ac21-41d6-873b-57d4be414f41 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:34.818885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:34.837595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3f90-85cf-4937-8e9c-4e525d081590 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:34.846910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.847997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.848440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.848927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.852267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:34.924920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:34.926264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:34.926780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:34.927331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:34.928934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:34.933775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:34.984254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 937/1890] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:34 2022] POST /identity/v3/projects => generated 369 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:34.987229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:34.991795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.017054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.020718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc1d81-aa0d-42f9-ac48-3ff9066b0543 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.029625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:35.046797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 954/1891] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:35.049850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.051193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.051886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.106649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:35.107695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.209527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adee320-727e-4935-ad1f-a949afcbf58a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.235192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.236748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.237628 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.238334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] PATH_INFO: `/v3/users/6a048da3552141aa941b7aa2ed191928` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.242404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a048da3552141aa941b7aa2ed191928)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.246933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.285611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 955/1892] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:35.285611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.285611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.310677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8f286-5ba6-48b3-9e48-9baa40215239 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.322757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.324422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.325285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.326283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.328363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.332693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f72ded-f99c-4b32-acd1-bb80d56b798a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.344692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a048da3552141aa941b7aa2ed191928'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:35.367748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a048da3552141aa941b7aa2ed191928'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:35.385183 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a048da3552141aa941b7aa2ed191928'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:35.390246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 956/1893] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:35 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:35.390246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.390246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.390246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.430950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a048da3552141aa941b7aa2ed191928'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:35.432095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:35.432259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a048da3552141aa941b7aa2ed191928'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:35.435337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.453564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 957/1894] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:35.456002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.474906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 938/1895] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:34 2022] DELETE /identity/v3/users/6a048da3552141aa941b7aa2ed191928 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:35.474906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.486061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.533587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:35.540628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3bbed-9245-4425-bc10-e7e570825e78 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.665499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dace474-78da-4101-ae0e-f8894c1ff9f0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.682321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.686181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.687082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.690088 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.692324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:35.693302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.696892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.723079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 939/1896] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:35.731339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.736125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.737661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.738277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.739160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.741199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.745574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.778259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 958/1897] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:35.780958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.790472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051291-a069-40ab-8f86-7a9619934cb5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.804309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.808957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.814746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.815397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.817901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.818310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9b159-87ed-479f-93e8-48ba10601690 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.823842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.824305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.825203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.825786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.826274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.827212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.829575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.852249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 959/1898] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:35.853865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:35.858857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:35.882687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389554de-6d00-48ee-928c-b1e964f6bb54 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:35.891171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:35.893449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:35.893926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:35.894267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] PATH_INFO: `/v3/projects/3e24e8cd4aa84e75b511d18e6ac7c197` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:35.897403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e24e8cd4aa84e75b511d18e6ac7c197)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:35.901441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:35.941125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 940/1899] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:35 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:35.943564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:36.059043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e24e8cd4aa84e75b511d18e6ac7c197', 'request_id': 'req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.071232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:36.079479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e24e8cd4aa84e75b511d18e6ac7c197', 'request_id': 'req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.102884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e24e8cd4aa84e75b511d18e6ac7c197', 'request_id': 'req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.160486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] The token cache is being invalidate because project 3e24e8cd4aa84e75b511d18e6ac7c197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:36.161213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.162557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.166743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 960/1900] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:35 2022] DELETE /identity/v3/projects/3e24e8cd4aa84e75b511d18e6ac7c197 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:36.169301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:36.172435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:36.175165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90514b92-9952-4382-be8e-cf660277b6a9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:36.308862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:36.310815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:36.311257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:36.311740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] PATH_INFO: `/v3/projects/ffa06a8a68a445ee8eb80bf47e89f3c6/users/2774b39da15b49fca49583e16457a5b5/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:36.322014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8417f3-e8a2-46f4-828f-b413fda1e9e7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:36.328180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:36.328897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:36.329242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:36.329570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:36.330971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:36.332695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:36.343312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa06a8a68a445ee8eb80bf47e89f3c6, user_id=2774b39da15b49fca49583e16457a5b5, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:36.349162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:36.380917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 961/1901] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:36 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:36.382631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:36.387130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:36.394783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 941/1902] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:35 2022] PUT /identity/v3/projects/ffa06a8a68a445ee8eb80bf47e89f3c6/users/2774b39da15b49fca49583e16457a5b5/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:36.396229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:36.398925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:36.420933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de165-5983-45a6-8bc2-ccc4ff19e7a8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:36.438725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:36.534513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d94d-a2c1-46bd-9236-1f134395c126 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:36.547844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:36.549602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:36.550250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:36.551023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] PATH_INFO: `/v3/projects/d5fd7ab2f160465bb87364c513e48f29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:36.553645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fd7ab2f160465bb87364c513e48f29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:36.557907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:36.591185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:36.593759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:36.594733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:36.595636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] PATH_INFO: `/v3/users/4ff8ea0aa3924316aad19da48046cf59` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:36.598944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff8ea0aa3924316aad19da48046cf59)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:36.604668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:36.654199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fd7ab2f160465bb87364c513e48f29', 'request_id': 'req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.668562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fd7ab2f160465bb87364c513e48f29', 'request_id': 'req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.699799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fd7ab2f160465bb87364c513e48f29', 'request_id': 'req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.738294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff8ea0aa3924316aad19da48046cf59'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.739076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] The token cache is being invalidate because project d5fd7ab2f160465bb87364c513e48f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:36.739528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.747092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.747458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 942/1903] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:36 2022] DELETE /identity/v3/projects/d5fd7ab2f160465bb87364c513e48f29 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:36.749366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:36.754349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:36.758815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff8ea0aa3924316aad19da48046cf59'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.759967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff8ea0aa3924316aad19da48046cf59'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.791535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff8ea0aa3924316aad19da48046cf59'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.794471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a988c8d-b6e3-4d77-be76-8a705f5e7a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff8ea0aa3924316aad19da48046cf59'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:36.822610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=11,keystone:SELECT=84,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:36.857258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 962/1904] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:36 2022] DELETE /identity/v3/users/4ff8ea0aa3924316aad19da48046cf59 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:36.858918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701035-5752-4793-b428-b7e1f391612c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:36.859421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701035-5752-4793-b428-b7e1f391612c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:36.859809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701035-5752-4793-b428-b7e1f391612c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:36.923758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3c701035-5752-4793-b428-b7e1f391612c None None] MFA Rules not processed for user `2774b39da15b49fca49583e16457a5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:36.926371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3c701035-5752-4793-b428-b7e1f391612c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:36.932327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb187d5-23fa-49db-95bd-d1d0b0ff2853 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.142758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:37.167416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 963/1905] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:37.167416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.167416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.167416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.169649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.170720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.171269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.171691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.172581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.174584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.190979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 943/1906] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:37.196367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.197652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:37.199782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.200849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.218526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 964/1907] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:37.220101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.222034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.239575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634e1bb-1767-4184-be83-12c71fde7312 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.246971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.248804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.249216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.249711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.250908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.252823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098e4c4-acb5-4a26-bca2-f7ca6161eb03 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.272739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 965/1908] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:37.274753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.275380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.275770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.314444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:37.317311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.339959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 966/1909] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:37.341731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.363446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d39b7-642a-42ee-846c-67d1be29be31 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.399052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.399708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c11e3-682d-4f24-a8b2-0bcc0e2268a0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.401006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.402515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.403062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] PATH_INFO: `/v3/projects/ec799b69172c4dec8f7aeaf413ac2948/users/6e505377317a4fbfaa7d0dcb86097103/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.408872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.410157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.410727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.411258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.427595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.430244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.459261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec799b69172c4dec8f7aeaf413ac2948, user_id=6e505377317a4fbfaa7d0dcb86097103, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.464652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.471721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 944/1910] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:37.473501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.515598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0c42a-4b8b-4e50-8e43-bf3044872419 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.526761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 967/1911] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:37 2022] PUT /identity/v3/projects/ec799b69172c4dec8f7aeaf413ac2948/users/6e505377317a4fbfaa7d0dcb86097103/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:37.526761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.526761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.682254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.682254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.682254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.682254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.682949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:37.682949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.682949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc7b90-cc33-4dd2-a775-bcaea7e9ca04 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.704643 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398fdce-527c-4f14-9304-8b6a0f52c1f6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.714601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 945/1912] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:37.716412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.716920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.717203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.731545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:37.733075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:37.733525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:37.733987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] PATH_INFO: `/v3/users/2774b39da15b49fca49583e16457a5b5` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:37.736902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2774b39da15b49fca49583e16457a5b5)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:37.741631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:37.766966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] MFA Rules not processed for user `6e505377317a4fbfaa7d0dcb86097103`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:37.769022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.815736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:37.844852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2774b39da15b49fca49583e16457a5b5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:37.860740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2774b39da15b49fca49583e16457a5b5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:37.862245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2774b39da15b49fca49583e16457a5b5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:37.876417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2774b39da15b49fca49583e16457a5b5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:37.877732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2774b39da15b49fca49583e16457a5b5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:37.895884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 946/1913] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:37.897401 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.900111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.915339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 968/1914] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:37 2022] DELETE /identity/v3/users/2774b39da15b49fca49583e16457a5b5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:37.920160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:37.921976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:37.923754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779b7c-c0c3-4819-a4c1-fd950db03aa2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:37.960370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:38.026025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.027629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.028146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.028770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] PATH_INFO: `/v3/projects/ef072b299931436dae8a9635e8dd08cb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.031111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef072b299931436dae8a9635e8dd08cb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.034944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.038379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef857a-d7bd-4a2e-b8b0-54b357aed2c6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.046234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.047337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.047823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.048297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.049816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.053681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.094394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 969/1915] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:37 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:38.095889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.101649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.113290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef072b299931436dae8a9635e8dd08cb', 'request_id': 'req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:38.125983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef072b299931436dae8a9635e8dd08cb', 'request_id': 'req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:38.140006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef072b299931436dae8a9635e8dd08cb', 'request_id': 'req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:38.162562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] The token cache is being invalidate because project ef072b299931436dae8a9635e8dd08cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:38.163322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:38.164219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ae6cc35a-d297-46b0-a7bd-768a8fe3ad81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:38.166967 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 947/1916] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:37 2022] DELETE /identity/v3/projects/ef072b299931436dae8a9635e8dd08cb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:38.169582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.170005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.170679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.178985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4bfe4-d4af-40d2-a692-e89a1ef57543 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.202201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:38.204331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.274092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.275223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.275726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.276253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.279489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.289964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:38.313899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 948/1917] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:38.314983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.316676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.331161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befcd95-504c-47f4-a328-6b083d3a4745 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.337298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.338188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.338682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.339179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.340538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.344546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5c01d-f5be-4d74-be4c-8d824b9cc7c7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.380709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 949/1918] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/users => generated 291 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:38.382520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.382813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.383091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.407688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.410054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.413527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:38.415757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.423849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 970/1919] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:38.424763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.427205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.431406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 950/1920] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:38.433110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.451488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb2b8c-ad5a-4e94-aa72-a9d7ccaf28cb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.462548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.463360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.463784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.464170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.465014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.466855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d2a10-d166-4cc3-9bd6-ed81b7328211 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.491188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 951/1921] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:38.493727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.494899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.495530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.537439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:38.539507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:38.551027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 952/1922] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:38.551889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bca59a-22e5-449e-bafc-78fd7c1081f7 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.552280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.566169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9bf80-bc73-4aad-86ee-2e908575baed None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.570845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.572955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.573946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.574161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.574384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.574711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.574908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.575543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.575949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.578072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.582007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:38.595586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.597656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.603582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 953/1923] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:38.605140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.622165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 971/1924] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:38.626439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.627523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db14c-f14f-45a2-a1db-a1e4eb5eb747 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.637487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.639221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.640336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.641045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.642843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:38.643931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a274-5db5-4d9d-b21a-3bcfd75e2c40 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.644211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.655976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.659685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:38.660940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.661467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.661856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] PATH_INFO: `/v3/projects/dedd47ee47584721a44b706982007322/users/9712a83bee1144aba9f807a220ed4ca4/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.685876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedd47ee47584721a44b706982007322, user_id=9712a83bee1144aba9f807a220ed4ca4, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:38.688774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d18c9-ad9d-4948-a7d7-9f8bc4bfa19d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:38.716916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 954/1925] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:38.722794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 972/1926] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:38 2022] PUT /identity/v3/projects/dedd47ee47584721a44b706982007322/users/9712a83bee1144aba9f807a220ed4ca4/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:38.724218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:38.737499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:38.738540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:38.739608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:38.756359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283ab69-e9ff-4c27-a4df-b362d7f7b98f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:38.901971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] MFA Rules not processed for user `9712a83bee1144aba9f807a220ed4ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:38.910266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:39.004770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.010383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.011079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.011561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.017003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.032634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.218084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 955/1927] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/projects => generated 365 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:39.224936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.284326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 973/1928] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:39.291689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.348954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471a3b6-9fe3-4ecf-9b8e-57f914d48e94 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.363981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afb89c-cbcc-4a33-a03b-1779bd655620 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.374842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.376209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.376805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.377434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.378759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.379201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.383505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.385955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.386452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.387038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.388621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.392878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.426536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 956/1929] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:39 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:39.446444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.450533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:39.478269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e7f27-d823-49d5-9b1b-e36b8f38befa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.500869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.501294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 974/1930] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:39 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:39.503165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.504919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.506152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.506670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.508558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.511439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.580568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 957/1931] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:39.580568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.584756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:39.633295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093f947-9c76-45d7-a777-69e6f39278f1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.712432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593b30-a378-4a1b-a1f1-671917496569 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.727960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.729326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.729760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.730175 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.732805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.736112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.758948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:39.775942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 975/1932] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:39 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:39.778274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.799525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912327a5-0f32-4809-8b2e-00007fa93561 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.801504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.804263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.804726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.805162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] PATH_INFO: `/v3/projects/ffa06a8a68a445ee8eb80bf47e89f3c6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.806838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa06a8a68a445ee8eb80bf47e89f3c6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.807495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.808843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.809682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.810188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.810425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.812165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.815238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.845620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 976/1933] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:39.845620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.867646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c3e25-7fdc-4235-ab0f-655d39f6123e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.877451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:39.878309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:39.878697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:39.879018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:39.879970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:39.882933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:39.911157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 977/1934] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:39.912379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.930001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa06a8a68a445ee8eb80bf47e89f3c6', 'request_id': 'req-73a56a78-25d3-4da2-9bc0-6d74a8afc856'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:39.930818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f05b8-7264-4145-ba5d-ead96e89a3e7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:39.949886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa06a8a68a445ee8eb80bf47e89f3c6', 'request_id': 'req-73a56a78-25d3-4da2-9bc0-6d74a8afc856'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:39.963362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa06a8a68a445ee8eb80bf47e89f3c6', 'request_id': 'req-73a56a78-25d3-4da2-9bc0-6d74a8afc856'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:39.988600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] The token cache is being invalidate because project ffa06a8a68a445ee8eb80bf47e89f3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:39.989123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:39.990765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:39.999653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 958/1935] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:39 2022] DELETE /identity/v3/projects/ffa06a8a68a445ee8eb80bf47e89f3c6 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:39.999653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:39.999653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:40.081019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:40.084549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.085129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.085855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] PATH_INFO: `/v3/projects/51de86ebb2484e339f1ea331732d2b8b/users/1e815d36636844e28d4decf8176e45da/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.119179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51de86ebb2484e339f1ea331732d2b8b, user_id=1e815d36636844e28d4decf8176e45da, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:40.124602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5918539-4838-4651-b3d6-25104112f46f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:40.150180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56a78-25d3-4da2-9bc0-6d74a8afc856 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:40.154794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 978/1936] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:39 2022] PUT /identity/v3/projects/51de86ebb2484e339f1ea331732d2b8b/users/1e815d36636844e28d4decf8176e45da/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:40.157578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.158321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.159068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.197284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:40.200605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:40.278860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:40.281993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.283075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.284012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] PATH_INFO: `/v3/projects/f49d1f019f5d43dba41863a50942cd06/users/908f431989a941e3a2fa095160c37faf/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.315923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:40.366367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49d1f019f5d43dba41863a50942cd06, user_id=908f431989a941e3a2fa095160c37faf, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:40.372440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d699eac-3c00-4b9e-9747-bdd712a2c690 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:40.404025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 959/1937] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:39 2022] PUT /identity/v3/projects/f49d1f019f5d43dba41863a50942cd06/users/908f431989a941e3a2fa095160c37faf/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:40.410137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.410611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.410985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.412414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 979/1938] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:40.419790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:40.453487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae562ac-86b1-4d16-bdef-a4100d05daf2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:40.454072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:40.479432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] MFA Rules not processed for user `1e815d36636844e28d4decf8176e45da`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:40.483143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:40.635518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:40.637530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.638290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.638998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.640867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:40.645301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d06e-542e-4123-b9dc-a90ba2645a38 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:40.673488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 980/1939] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:40.676625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.677401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.677971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.721518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:40.724251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:40.759979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 981/1940] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:40.759979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:40.797948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8035b8-1728-4621-b457-4897d43427fa None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:40.818163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 960/1941] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:40.819045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:40.821940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:40.823239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.825164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.826801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] PATH_INFO: `/v3/projects/f49d1f019f5d43dba41863a50942cd06/users/908f431989a941e3a2fa095160c37faf/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.835994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49d1f019f5d43dba41863a50942cd06, user_id=908f431989a941e3a2fa095160c37faf, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:40.849096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:40.887005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26dc22-4ce0-41a3-81d1-67083b7a515d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:40.910548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:40.912171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:40.914169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:40.918138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:40.921500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:40.922530 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:40.930507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc915-8423-4f3e-94bc-6a6a17a0a5cb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:40.959961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 982/1942] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:40 2022] PUT /identity/v3/projects/f49d1f019f5d43dba41863a50942cd06/users/908f431989a941e3a2fa095160c37faf/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:40.962890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:40.973911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:41.030194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 961/1943] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:41.042047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.042047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.042047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.126259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] MFA Rules not processed for user `908f431989a941e3a2fa095160c37faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:41.131114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:41.330386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172c76-3b94-47b3-9f72-18169c872ee1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.361621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.361621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.361621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.361621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.362409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.366112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.428593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 962/1944] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:41.445697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.475152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd590-40a4-4d93-99e4-ec6b1933ed4d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.484900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.486877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.488225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.488865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.490983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.495847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.501415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 983/1945] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:40 2022] POST /identity/v3/projects => generated 365 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:41.501415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.506800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:41.583505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 963/1946] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:41 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:41.585474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.674646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:41.688132 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ae673-232a-491e-a27a-4f84fe5459ae admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.702294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:41.733480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dff99-1f30-4d37-9e6f-c9461295891d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.743647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.745002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.745560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.746130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.747610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.751756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.800391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 964/1947] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:41 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:41.800391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.811156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.813136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.813136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.813852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.818988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ea9e1-48b2-4d58-a8c9-22fd519129e6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.830434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.831440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.831948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.836097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.836097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.841363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.884683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 965/1948] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:41 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:41.886413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.915914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c5dc-11d2-46f0-ad21-10b2081ccd1a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.929547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.930973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.931488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.932249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.933399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.936683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.964138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 966/1949] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:41.966118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:41.983003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:41.985562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:41.986202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb0061-4ec1-416d-b763-e421313c4e7d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:41.995904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:41.997233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:41.997796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:41.998277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:41.999757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.000704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 984/1950] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:42.001889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.003668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.018872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b6772-d952-4df4-809f-8c4e4326282d service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.034098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.035602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.037004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 967/1951] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:42.042073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.042961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] PATH_INFO: `/v3/projects/59678df2650241cab7bc83c500e7c382/users/1fc7c04124c44749a871b6027b6ae8a5/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.045023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.071191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62fc8a-4f3c-4af1-8baf-1db89ffedf34 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.079358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59678df2650241cab7bc83c500e7c382, user_id=1fc7c04124c44749a871b6027b6ae8a5, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.086753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.089539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.090483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.090709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.091836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.093998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.098014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.123460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 985/1952] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:42 2022] PUT /identity/v3/projects/59678df2650241cab7bc83c500e7c382/users/1fc7c04124c44749a871b6027b6ae8a5/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:42.125311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 968/1953] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:42.131227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.131547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.163805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb47b0-2953-46d4-9c0a-7c5506a90425 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.294582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600299-9799-4708-8ff5-238ac4ec964d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.304505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.305999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.306707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.307331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.308862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.312356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.345381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.345381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.346112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 986/1954] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:42.346263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.346263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] PATH_INFO: `/v3/projects/1b1fba96612b4ed0ae53b43cdd4e3528/users/bb3927ae30c54149974d3037c1022854/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.346579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.371927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630b24-960e-4335-9443-a05a910c0cd7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.380824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.383800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.385803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.387913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] PATH_INFO: `/v3/projects/59678df2650241cab7bc83c500e7c382/users/1fc7c04124c44749a871b6027b6ae8a5/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.392075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59678df2650241cab7bc83c500e7c382, user_id=1fc7c04124c44749a871b6027b6ae8a5, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.397430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647184e1-2943-4a85-860c-4446912be28b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.402072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1fba96612b4ed0ae53b43cdd4e3528, user_id=bb3927ae30c54149974d3037c1022854, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.412020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2966d-cdb2-4f19-a6d6-f5fc507e17a2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:42.439423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 987/1955] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:42 2022] PUT /identity/v3/projects/59678df2650241cab7bc83c500e7c382/users/1fc7c04124c44749a871b6027b6ae8a5/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:42.466849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 969/1956] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:42 2022] PUT /identity/v3/projects/1b1fba96612b4ed0ae53b43cdd4e3528/users/bb3927ae30c54149974d3037c1022854/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:42.471608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.472073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.472485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.486163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.486700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.487209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.534858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:42.540477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:42.544386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:42.560398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] MFA Rules not processed for user `1fc7c04124c44749a871b6027b6ae8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:42.563038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:42.723906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 970/1957] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:42.745123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.747876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:42.794077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:42.859928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 988/1958] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:42.886695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:42.889744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:42.950668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e34783-7668-42cb-80e1-8cb716a45f90 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:42.963369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:42.965586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:42.966138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:42.967557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:42.994946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:42.998799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:43.025543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 971/1959] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:43.087757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473db1c-2b39-4dfb-b965-00e1b0b9051c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:43.098729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:43.100121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:43.100572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:43.101131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:43.151165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:43.156505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd716c3-c5c2-45ae-a6c4-96004ca74fa7 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:43.176594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 989/1960] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:43.218232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:43.248103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84da6-eaa6-409b-916a-c392b4dfa42e service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:43.261301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:43.263242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:43.263811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:43.265902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] PATH_INFO: `/v3/users/9712a83bee1144aba9f807a220ed4ca4` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:43.268772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9712a83bee1144aba9f807a220ed4ca4)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:43.273152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:43.413677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9712a83bee1144aba9f807a220ed4ca4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:43.455546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9712a83bee1144aba9f807a220ed4ca4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:43.487691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9712a83bee1144aba9f807a220ed4ca4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:43.549305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9712a83bee1144aba9f807a220ed4ca4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:43.551144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9712a83bee1144aba9f807a220ed4ca4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:43.574074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 972/1961] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:43 2022] DELETE /identity/v3/users/9712a83bee1144aba9f807a220ed4ca4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:43.582751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:43.583458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:43.583936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:43.637787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:43.640196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:43.837394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 990/1962] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:43.851882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:43.855531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:43.996599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40a5d-244b-4120-b763-8514b8b5c52c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:44.005257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:44.006608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:44.007045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:44.008931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:44.030498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:44.045406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7442ce-82ec-42f2-a095-6e42e9394b2b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:44.091061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 973/1963] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:44.327684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:44.331981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:44.356199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607728c-5fa7-46db-9d44-fc782c13f57e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:44.372522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:44.374558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:44.385465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:44.385465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] PATH_INFO: `/v3/users/1e815d36636844e28d4decf8176e45da` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:44.385465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e815d36636844e28d4decf8176e45da)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:44.385465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:44.472620 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:44.500456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e815d36636844e28d4decf8176e45da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:44.519590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e815d36636844e28d4decf8176e45da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:44.521445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e815d36636844e28d4decf8176e45da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:44.543013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e815d36636844e28d4decf8176e45da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:44.544492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e815d36636844e28d4decf8176e45da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:44.600636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 991/1964] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:44 2022] DELETE /identity/v3/users/1e815d36636844e28d4decf8176e45da => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:44.608085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231823-e190-4b54-9e70-bcff6039d232 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:44.608910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231823-e190-4b54-9e70-bcff6039d232 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:44.608910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231823-e190-4b54-9e70-bcff6039d232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:44.655687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-19231823-e190-4b54-9e70-bcff6039d232 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:44.658157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-19231823-e190-4b54-9e70-bcff6039d232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:44.730670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:44.867862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 974/1965] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:44.870847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:44.875066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:45.063439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad71db-f3a7-4cf2-bdbe-6592ea51ee8c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:45.074872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:45.076551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:45.077405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:45.078609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:45.117699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:45.122759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:45.143226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 992/1966] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:45.526406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:45.528898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:45.529828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:45.619507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] MFA Rules not processed for user `bb3927ae30c54149974d3037c1022854`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:45.622558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:45.801851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:45.806716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:45.881193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474774b6-2152-4502-a106-1d09a6cf9249 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:45.892192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:45.894427 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:45.895997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:45.899435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 975/1967] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:45.900238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] PATH_INFO: `/v3/projects/dedd47ee47584721a44b706982007322` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:45.903591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedd47ee47584721a44b706982007322)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:45.908778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:45.949901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:45.952598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:46.061502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedd47ee47584721a44b706982007322', 'request_id': 'req-c9253f28-4cb7-44a8-8573-ea31c14477ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.087310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedd47ee47584721a44b706982007322', 'request_id': 'req-c9253f28-4cb7-44a8-8573-ea31c14477ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.114878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedd47ee47584721a44b706982007322', 'request_id': 'req-c9253f28-4cb7-44a8-8573-ea31c14477ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.154916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] The token cache is being invalidate because project dedd47ee47584721a44b706982007322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:46.155416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.157016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.160682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 993/1968] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:45 2022] DELETE /identity/v3/projects/dedd47ee47584721a44b706982007322 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:46.162484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:46.165297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:46.171697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184081-7cfe-4d40-819c-6c3f0e9ce809 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:46.250213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:46.297286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:46.298558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:46.299046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:46.299589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:46.302827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:46.325072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9253f28-4cb7-44a8-8573-ea31c14477ed admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:46.334375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:46.335835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:46.336306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:46.336916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] PATH_INFO: `/v3/projects/51de86ebb2484e339f1ea331732d2b8b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:46.340918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=51de86ebb2484e339f1ea331732d2b8b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:46.343864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:46.359791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:46.464458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:46.466584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:46.495430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 976/1969] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:46.498909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:46.501804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:46.506417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51de86ebb2484e339f1ea331732d2b8b', 'request_id': 'req-be7045c9-59df-4141-a997-df775277a49a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.520944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51de86ebb2484e339f1ea331732d2b8b', 'request_id': 'req-be7045c9-59df-4141-a997-df775277a49a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.535159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51de86ebb2484e339f1ea331732d2b8b', 'request_id': 'req-be7045c9-59df-4141-a997-df775277a49a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.559134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c06f-7122-462e-bd98-73243ed415d7 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:46.565509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] The token cache is being invalidate because project 51de86ebb2484e339f1ea331732d2b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:46.565978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.567266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.570057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 994/1970] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:46 2022] DELETE /identity/v3/projects/51de86ebb2484e339f1ea331732d2b8b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:46.576872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:46.579311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:46.717638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7045c9-59df-4141-a997-df775277a49a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:46.726420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:46.728019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:46.728526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:46.728988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] PATH_INFO: `/v3/users/908f431989a941e3a2fa095160c37faf` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:46.731564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908f431989a941e3a2fa095160c37faf)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:46.735961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:46.745488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:46.747641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:46.747900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:46.748265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] PATH_INFO: `/v3/users/1fc7c04124c44749a871b6027b6ae8a5` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:46.754476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc7c04124c44749a871b6027b6ae8a5)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:46.758957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:46.883780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908f431989a941e3a2fa095160c37faf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.885146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc7c04124c44749a871b6027b6ae8a5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.902422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908f431989a941e3a2fa095160c37faf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.904170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc7c04124c44749a871b6027b6ae8a5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.906135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc7c04124c44749a871b6027b6ae8a5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.922654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908f431989a941e3a2fa095160c37faf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.926158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc7c04124c44749a871b6027b6ae8a5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.930877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfea7ae1-ca2a-4338-bf9d-dcbdec1fc207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc7c04124c44749a871b6027b6ae8a5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.945653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:46.970059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908f431989a941e3a2fa095160c37faf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.971314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908f431989a941e3a2fa095160c37faf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:46.989034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 977/1971] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:46 2022] DELETE /identity/v3/users/908f431989a941e3a2fa095160c37faf => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:46.990497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 995/1972] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:46 2022] DELETE /identity/v3/users/1fc7c04124c44749a871b6027b6ae8a5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:48.313385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:48.316311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:48.458787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:48.489324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6ec1-853b-4a99-8996-28844a95f8f5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:48.490527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6ec1-853b-4a99-8996-28844a95f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:48.490752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6ec1-853b-4a99-8996-28844a95f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:48.522174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c897-3b78-4f57-9a8b-27be936ff543 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:48.527354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-25bb6ec1-853b-4a99-8996-28844a95f8f5 None None] MFA Rules not processed for user `bb3927ae30c54149974d3037c1022854`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:48.530151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-25bb6ec1-853b-4a99-8996-28844a95f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:48.619788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:48.621663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:48.622310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:48.622866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] PATH_INFO: `/v3/projects/59678df2650241cab7bc83c500e7c382` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:48.625043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59678df2650241cab7bc83c500e7c382)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:48.628939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:48.650279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 996/1973] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:48.721795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59678df2650241cab7bc83c500e7c382', 'request_id': 'req-47b3ee84-e1d5-4953-879d-e97755968631'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:48.733514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59678df2650241cab7bc83c500e7c382', 'request_id': 'req-47b3ee84-e1d5-4953-879d-e97755968631'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:48.750708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59678df2650241cab7bc83c500e7c382', 'request_id': 'req-47b3ee84-e1d5-4953-879d-e97755968631'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:48.772097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] The token cache is being invalidate because project 59678df2650241cab7bc83c500e7c382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:48.772891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:48.774025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:48.776896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 978/1974] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:48 2022] DELETE /identity/v3/projects/59678df2650241cab7bc83c500e7c382 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:48.784432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a643-ee7f-4af2-9909-a7fb48be7943 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:48.784893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a643-ee7f-4af2-9909-a7fb48be7943 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:48.785424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a643-ee7f-4af2-9909-a7fb48be7943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:48.820552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1772a643-ee7f-4af2-9909-a7fb48be7943 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:48.822209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1772a643-ee7f-4af2-9909-a7fb48be7943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:48.848904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:48.850786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:48.936019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 997/1975] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:48.937674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3ee84-e1d5-4953-879d-e97755968631 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:48.945462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:48.946590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:48.946888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:48.947171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] PATH_INFO: `/v3/projects/f49d1f019f5d43dba41863a50942cd06` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:48.948542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:48.948874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:48.949255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:48.949745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f49d1f019f5d43dba41863a50942cd06)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:48.952353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:49.008052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:49.011592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.021221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:49.042818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 998/1976] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:49.048536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.061599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f49d1f019f5d43dba41863a50942cd06', 'request_id': 'req-89688cc5-f2f6-40bd-a750-226c8466a6f5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.068147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd12150-6fb4-4f8f-bd5a-2d132d5f811f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:49.074443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f49d1f019f5d43dba41863a50942cd06', 'request_id': 'req-89688cc5-f2f6-40bd-a750-226c8466a6f5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.086223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f49d1f019f5d43dba41863a50942cd06', 'request_id': 'req-89688cc5-f2f6-40bd-a750-226c8466a6f5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.117919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] The token cache is being invalidate because project f49d1f019f5d43dba41863a50942cd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:49.118895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.120581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-89688cc5-f2f6-40bd-a750-226c8466a6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.125455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 979/1977] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:48 2022] DELETE /identity/v3/projects/f49d1f019f5d43dba41863a50942cd06 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:49.131652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.132869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.134239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.162941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:49.166002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.189330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:49.190366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.190923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.191481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.192502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:49.193116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:49.195588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:49.213317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 999/1978] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:25:49.221099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.223918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.269153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a10f99-ce03-4b52-952c-c3273efa12b5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:49.279842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:49.280951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.281573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.282444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.284235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:49.289814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f1819-900b-4f59-ac1e-d693cfc8e9c6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:49.344317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:49.347562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 980/1979] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:49.349218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.355764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.370540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1000/1980] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:49 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:49.382343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.382343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.382343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.389835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e91284-f54d-4946-bb9f-6d68f30de3b1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:49.448618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:49.451043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.621521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:49.623649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.624398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.624669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] PATH_INFO: `/v3/users/6e505377317a4fbfaa7d0dcb86097103` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.626991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e505377317a4fbfaa7d0dcb86097103)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:49.629207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:49.744582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1001/1981] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:49.758601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.799769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138826f1-20e9-4fef-9be0-188ca965a8bc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:49.820462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e505377317a4fbfaa7d0dcb86097103'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.828069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:49.828859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:49.829141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:49.837855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:49.839921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:49.849476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:49.878484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e505377317a4fbfaa7d0dcb86097103'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.889653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:49.903629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e505377317a4fbfaa7d0dcb86097103'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.948790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e505377317a4fbfaa7d0dcb86097103'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.949756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e505377317a4fbfaa7d0dcb86097103'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:49.952695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1002/1982] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:49 2022] POST /identity/v3/users => generated 278 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:49.953708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.964447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 981/1983] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:49 2022] DELETE /identity/v3/users/6e505377317a4fbfaa7d0dcb86097103 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:49.968936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.971333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:49.976559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:49.990769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d448-81dd-482d-9536-bfe11f3890a6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.109359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66198d-5adc-401e-9c9d-c847c48f955e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.116265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.117205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.117563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.117821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.120002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.120311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.120457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.120457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.120786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff72b7b-0af8-46d7-8a87-b2810160486c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.121253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.123013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:50.123545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.129568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.138371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 982/1984] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:50.140374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.140861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.141322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.170239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:50.171942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:50.172281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1003/1985] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:50.173780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.188396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 983/1986] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:50.188396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.201774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e323bd-2a71-46e0-a16c-cca82df8883e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.207646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cd86b-3779-4189-af23-07cc95b5f5f7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.208923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.209690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.209980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.210261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.211176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.213663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.218010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.219342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.220071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.220809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.226444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.230492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.249369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1004/1987] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:50.251309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.258387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:50.259246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 984/1988] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:50.261151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.378403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b824a0-f1c2-456a-b13d-d52ebb8f01a5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.380414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba0b6c-163f-44d6-a8df-ea2ce3e2c1b9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.384446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.385692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.386154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.386831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] PATH_INFO: `/v3/projects/40df514be5104c5d868870c969214dc5/users/1e3b6b4d699b4038b281ae3ae2b10163/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.405680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:50.421501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=40df514be5104c5d868870c969214dc5, user_id=1e3b6b4d699b4038b281ae3ae2b10163, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.426853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.480101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 985/1989] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:50 2022] PUT /identity/v3/projects/40df514be5104c5d868870c969214dc5/users/1e3b6b4d699b4038b281ae3ae2b10163/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:50.489863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.528666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.529589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.530046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.530476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.554551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.557106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e67e27-a8e1-4163-b5ac-47e1d222bc0b service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.572717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1005/1990] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:50.574579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.575004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.575369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.623957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:50.655574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] MFA Rules not processed for user `1e3b6b4d699b4038b281ae3ae2b10163`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:50.657836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:50.748955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff0cd2-0c55-4e8b-850c-cdf13d636eab admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.784543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.787329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.789942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.793751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.795108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.802281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.822297 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1006/1991] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:50.831766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.853722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8253e4-30bf-4db4-ba84-681594f3227e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.863432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.864520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.865358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.865997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.867401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.869868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.880971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 986/1992] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/users => generated 292 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:50.887921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.913756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b417bc-74e7-412f-9327-0ea8c1062a04 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:50.925070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:50.929624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:50.929624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:50.929624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:50.929624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:50.931445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1007/1993] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/projects => generated 353 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:50.934573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:50.934834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:50.986601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 987/1994] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:50.990962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.014838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b077a-e218-4db1-8676-4c161828beb5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.252417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1ba36-703a-40c9-a071-d790d83027d2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.274734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.275411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.275723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.275997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.276721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.286144 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.331625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1008/1995] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:50 2022] POST /identity/v3/users => generated 283 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:51.336522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.345431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.350509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.362197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.364170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.366036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.367353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5a2f-4aaa-421e-9515-1271755e8942 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.374165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.403080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.406323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.406711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.408607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.409567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.414172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.421787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 988/1996] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:51.424067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.427214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:51.446671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1009/1997] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:51.447977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.472819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53c3b-b1a8-4e15-ad0d-c254878fe02b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.484204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.485367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.485655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.485939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] PATH_INFO: `/v3/projects/0df93b27f35b40a488e3f8c636fde300/users/d56451a43ab34353bc60fbb3481df548/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.490849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:51.508387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45cfee-54a0-4c5e-a3f4-bdaa8f59f75f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.529852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df93b27f35b40a488e3f8c636fde300, user_id=d56451a43ab34353bc60fbb3481df548, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.537952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.545015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.547079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.547469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.554642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] PATH_INFO: `/v3/projects/ec799b69172c4dec8f7aeaf413ac2948` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.567158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec799b69172c4dec8f7aeaf413ac2948)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.569362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.583576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1010/1998] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:51 2022] PUT /identity/v3/projects/0df93b27f35b40a488e3f8c636fde300/users/d56451a43ab34353bc60fbb3481df548/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:51.596563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.724248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec799b69172c4dec8f7aeaf413ac2948', 'request_id': 'req-47dbec82-da73-42d4-870c-efeda21edd8f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:51.737097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec799b69172c4dec8f7aeaf413ac2948', 'request_id': 'req-47dbec82-da73-42d4-870c-efeda21edd8f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:51.743864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4562933-3ae3-449b-9d79-59eefd2dad9b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:51.752622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec799b69172c4dec8f7aeaf413ac2948', 'request_id': 'req-47dbec82-da73-42d4-870c-efeda21edd8f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:51.783330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] The token cache is being invalidate because project ec799b69172c4dec8f7aeaf413ac2948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:51.783978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:51.784792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47dbec82-da73-42d4-870c-efeda21edd8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:51.792015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 989/1999] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:51 2022] DELETE /identity/v3/projects/ec799b69172c4dec8f7aeaf413ac2948 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:51.792015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b6f57-7059-4b22-8c2b-9b0bab2198a3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.792015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b6f57-7059-4b22-8c2b-9b0bab2198a3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.792015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b6f57-7059-4b22-8c2b-9b0bab2198a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.823706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0e4b6f57-7059-4b22-8c2b-9b0bab2198a3 None None] MFA Rules not processed for user `d56451a43ab34353bc60fbb3481df548`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:51.825258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0e4b6f57-7059-4b22-8c2b-9b0bab2198a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:51.875550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:51.886287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:51.891206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.891598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.892201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] PATH_INFO: `/v3/projects/cf44e760f7144d248c114b04c8a89841/users/d0698b2a215447a7aaa637c2264ee8fc/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.947603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf44e760f7144d248c114b04c8a89841, user_id=d0698b2a215447a7aaa637c2264ee8fc, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:51.958895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:51.977912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 990/2000] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:51.977912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:51.977912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:51.977912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:51.993665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1011/2001] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:51 2022] PUT /identity/v3/projects/cf44e760f7144d248c114b04c8a89841/users/d0698b2a215447a7aaa637c2264ee8fc/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:51.993665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:51.995862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.031343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:52.035793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.183071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3c906-065a-4d8d-8edd-1ef0db778b36 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.192771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 991/2002] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:52.194647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.200433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.201073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.201259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.201582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.201917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.202755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.210423 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.231222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6be24-9510-4b28-a7fc-0dedcf8acdad None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.241753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.242980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.243532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.244007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.245156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.248128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879df37-c7f5-4200-887e-435fc6d866c7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.251492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:52.266411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 992/2003] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:52.268922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.269447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.269888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.279885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1012/2004] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:51 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:52.279885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.296588 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:52.298362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.450296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 993/2005] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:52.451523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.456729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d62cb2-9ff9-4775-9625-2fd7337f515a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.471689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.471976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbdfe3-f18c-4ca5-87fe-c5a0a348bf4a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.475103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.475103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.475103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] PATH_INFO: `/v3/projects/cf44e760f7144d248c114b04c8a89841/users/d0698b2a215447a7aaa637c2264ee8fc/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.477189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf44e760f7144d248c114b04c8a89841, user_id=d0698b2a215447a7aaa637c2264ee8fc, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.481982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.510762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.511514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.511822 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.512105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.513187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.515431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1013/2006] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:52 2022] PUT /identity/v3/projects/cf44e760f7144d248c114b04c8a89841/users/d0698b2a215447a7aaa637c2264ee8fc/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:52.521617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f51a5-fb81-40ff-bbca-df1b76746532 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.525102 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.598184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 994/2007] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:52 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:52.606566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.606942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.610769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.669496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] MFA Rules not processed for user `d0698b2a215447a7aaa637c2264ee8fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:52.674546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.701145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:52.770923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c512c-ae4c-4006-9d21-37d1ed8e9c61 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.790098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.791209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.791677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.792120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.793808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:52.794344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.798049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.839674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1014/2008] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:52.839674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.852051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 995/2009] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:52 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:52.853671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.856709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:52.862301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74068da5-6603-4d91-b7ff-24107fdda67a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.871336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.874031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.874777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.875466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.877068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.878720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81ed6-6f3c-43a6-83fe-cc4edd1967bf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.880340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.884280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.885409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.885810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.886185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] PATH_INFO: `/v3/users/8b36e0160e4a486988965eb3ff1f9eaa` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.887872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b36e0160e4a486988965eb3ff1f9eaa)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.890108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.902037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1015/2010] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:52.908462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:52.931601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff84d9-67b6-4e8d-a0b7-106b4097aa5e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:52.944747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:52.946169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:52.946778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:52.947182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:52.950875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:52.957710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:52.968187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b36e0160e4a486988965eb3ff1f9eaa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:52.982519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b36e0160e4a486988965eb3ff1f9eaa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:53.001953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b36e0160e4a486988965eb3ff1f9eaa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:53.013857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1016/2011] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:52 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:53.015192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.017665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.034352 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b36e0160e4a486988965eb3ff1f9eaa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:53.035750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b36e0160e4a486988965eb3ff1f9eaa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:53.049967 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 996/2012] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:52 2022] DELETE /identity/v3/users/8b36e0160e4a486988965eb3ff1f9eaa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:53.052235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.055446 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.130352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52629a-c8f9-470b-8d99-7617bbc6076f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.150794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:53.205869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26f4c-830d-46a0-9205-84fec80b8e50 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.216366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.218504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.219706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.220450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] PATH_INFO: `/v3/projects/c54f18fc25434fad8c644fabd7d98aee/users/aec3e6f6d00244f1b1d77c79a35c1cf8/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.254363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54f18fc25434fad8c644fabd7d98aee, user_id=aec3e6f6d00244f1b1d77c79a35c1cf8, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.259364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.260688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.261294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.261901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.262221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.266707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.311102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 997/2013] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:53 2022] PUT /identity/v3/projects/c54f18fc25434fad8c644fabd7d98aee/users/aec3e6f6d00244f1b1d77c79a35c1cf8/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:53.316127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.319160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.343669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:53.412569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.415472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0e912-2f06-48a0-acf3-188c58715e4b service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.441434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1017/2014] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:53.441434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.441434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.441434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.499211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:53.504575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.547115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1018/2015] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:53.547115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.556484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19325045-7020-4fe2-8a3f-6e657c310802 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.567874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.569226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.569917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.570484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.571916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.576723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.585654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304155de-0aa0-4f38-b745-33ace267a7ad None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.619362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.629751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 998/2016] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:53 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:53.631429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.632767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.633272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.633810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.635701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.636513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.642252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.673641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1019/2017] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:53.675319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.701634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9093f8-c8a0-462d-8bd6-55713cd893dd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.719035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.720206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.720666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.721122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.726876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.729885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.761700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1020/2018] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:53.763798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.766782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:53.864793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73747f04-656b-47ff-82a7-e37cc17938c6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.885397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:53.885397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:53.885397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:53.885397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:53.905631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:53.909063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:53.936040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 999/2019] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:53.937610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:53.967864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabe73-123b-4cbf-9b16-2cf2e73990b2 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:53.997610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.001215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.001844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.002292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] PATH_INFO: `/v3/projects/c54f18fc25434fad8c644fabd7d98aee/users/aec3e6f6d00244f1b1d77c79a35c1cf8/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.013749 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c23cd1-043a-4858-ac11-db432d35921d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:54.014618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54f18fc25434fad8c644fabd7d98aee, user_id=aec3e6f6d00244f1b1d77c79a35c1cf8, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.020469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:54.022452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.026981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.027497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.029821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.038235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:54.053022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1000/2020] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:53 2022] PUT /identity/v3/projects/c54f18fc25434fad8c644fabd7d98aee/users/aec3e6f6d00244f1b1d77c79a35c1cf8/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:54.054557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:54.183689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:54.190140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e15068-d151-4fbe-821d-b44c51d6362c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:54.198984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.200176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.200661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.201131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.202564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.206727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9ca28-8e37-4f85-adff-5e90b11cdec6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:54.229645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.232962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1001/2021] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:54.236574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.236889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.237793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.239784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:54.267787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1021/2022] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:54.268587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:54.284219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44e3e-d0b7-4ba1-87cc-d11d858a3bab service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:54.292638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.293686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.293977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.294407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] PATH_INFO: `/v3/projects/ea07dc4bdd584fdc8e8bfb24b7f4655c/users/ebf5eb935fc84b4dbc3090186cae2144/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.320306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea07dc4bdd584fdc8e8bfb24b7f4655c, user_id=ebf5eb935fc84b4dbc3090186cae2144, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.323885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e852f5-e1f4-4444-8900-9d6bd25a5629 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:54.339422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] MFA Rules not processed for user `aec3e6f6d00244f1b1d77c79a35c1cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:54.341613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:54.354307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1022/2023] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:54 2022] PUT /identity/v3/projects/ea07dc4bdd584fdc8e8bfb24b7f4655c/users/ebf5eb935fc84b4dbc3090186cae2144/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:54.379132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:54.380195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.380680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.381150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.485765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] MFA Rules not processed for user `ebf5eb935fc84b4dbc3090186cae2144`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:54.488128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:54.596094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1002/2024] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:54.608577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:54.707956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1023/2025] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:25:54.714820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:54.883885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b9a78-9963-401e-bd74-5ef7b7be491b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:54.897389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badb562-b62b-4330-9872-8ec9b2581af2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:54.901992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.903187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.903667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.904144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.909570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:54.910781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:54.911252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:54.911705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:54.912966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.919187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:54.958056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:54.971429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.005772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1024/2026] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:54 2022] POST /identity/v3/projects => generated 373 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:55.008021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.021140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1003/2027] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:25:55.117625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:55.174896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d1838-a010-4526-a655-32f30f7d9245 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.183096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.184884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.185566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.186478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.187754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.191957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.239117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1025/2028] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:55 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:55.241410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.255907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.272163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8657-bb0a-41ea-8195-5183922d4827 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.280091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.281055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.281554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.282539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.283345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.285936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.305831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1004/2029] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:55.310989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.336473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3de0b4-d4f4-40ee-a293-ef360de5339e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.343389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.344746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.345232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.345896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] PATH_INFO: `/v3/projects/2c55d64432254c25bc1cc5c438bf7db2/users/493e1b1ac3e04569948d09ead82a20a4/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.374187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c55d64432254c25bc1cc5c438bf7db2, user_id=493e1b1ac3e04569948d09ead82a20a4, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.378743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.384420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46612d51-cc44-413b-9cc9-1a13ce44cf83 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.398158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.399437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.400736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.402202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.407560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:55.408903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1005/2030] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:55 2022] PUT /identity/v3/projects/2c55d64432254c25bc1cc5c438bf7db2/users/493e1b1ac3e04569948d09ead82a20a4/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:55.413324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.554757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.556702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.560476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d3a7-a866-45f2-8801-d72e8ebaafe0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.575175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.577252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.577765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.580058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.581288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.585652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1026/2031] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:55.586556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.586968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.591287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:55.614145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1006/2032] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:55.619153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc85f4-18a6-4b96-b97f-96d19ff5031e service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.624595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.626852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.628177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.628466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.628792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] PATH_INFO: `/v3/projects/1fe652f363784ab3bba71ea704a24b94` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.630777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe652f363784ab3bba71ea704a24b94)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.640774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.645027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021178dd-8768-4dcf-9cfc-82db58bd9338 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:55.650942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:55.651899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.652176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.652663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] PATH_INFO: `/v3/projects/2c55d64432254c25bc1cc5c438bf7db2/users/493e1b1ac3e04569948d09ead82a20a4/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.669211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c55d64432254c25bc1cc5c438bf7db2, user_id=493e1b1ac3e04569948d09ead82a20a4, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:55.674127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8d65c-3e22-4cb5-9a58-430dd3672943 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:55.681866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:55.694910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1007/2033] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:55 2022] PUT /identity/v3/projects/2c55d64432254c25bc1cc5c438bf7db2/users/493e1b1ac3e04569948d09ead82a20a4/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:55.709525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:55.710550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:55.711220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:55.756494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe652f363784ab3bba71ea704a24b94', 'request_id': 'req-f9cd699d-8286-45fc-993a-bc253f1ff265'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:55.779512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe652f363784ab3bba71ea704a24b94', 'request_id': 'req-f9cd699d-8286-45fc-993a-bc253f1ff265'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:55.785010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] MFA Rules not processed for user `493e1b1ac3e04569948d09ead82a20a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:55.787816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:55.799851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe652f363784ab3bba71ea704a24b94', 'request_id': 'req-f9cd699d-8286-45fc-993a-bc253f1ff265'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:55.854234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] The token cache is being invalidate because project 1fe652f363784ab3bba71ea704a24b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:55.860597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:55.862161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:55.867687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1027/2034] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:55 2022] DELETE /identity/v3/projects/1fe652f363784ab3bba71ea704a24b94 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:55.880609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:55.884000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:55.909868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:56.090273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd699d-8286-45fc-993a-bc253f1ff265 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:56.097647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1008/2035] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:56.104268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:56.110883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:56.111533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:56.112371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] PATH_INFO: `/v3/users/200698f0d3004db4a22cea4cbb52754d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:56.116534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=200698f0d3004db4a22cea4cbb52754d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:56.128857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:56.174555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:56.176893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:56.284914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200698f0d3004db4a22cea4cbb52754d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:56.307231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200698f0d3004db4a22cea4cbb52754d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:56.309832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200698f0d3004db4a22cea4cbb52754d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:56.331879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200698f0d3004db4a22cea4cbb52754d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:56.332534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200698f0d3004db4a22cea4cbb52754d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:56.395238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1028/2036] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:55 2022] DELETE /identity/v3/users/200698f0d3004db4a22cea4cbb52754d => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:56.450228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5ff7-c3f8-441c-8a69-71343a181f7c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:56.595310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:56.596017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:56.596304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:56.596583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:56.603432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:56.769228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:56.783631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:56.785452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:56.786684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:56.789757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:56.815518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1009/2037] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:57.076942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c204cd-2e5b-494f-9856-85bc4df20a11 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:57.089691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:57.090892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:57.091270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:57.091550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] PATH_INFO: `/v3/users/d56451a43ab34353bc60fbb3481df548` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:57.095625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56451a43ab34353bc60fbb3481df548)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:57.099030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:57.230021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:57.231778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:57.247243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:57.278257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec33ee-2ac0-4b29-84be-25287cb2fc30 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:57.295730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:57.297250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:57.298676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:57.299637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] PATH_INFO: `/v3/projects/a6ed080a9be74314800fc54d575464bb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:57.313152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ed080a9be74314800fc54d575464bb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:57.320200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:57.336020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56451a43ab34353bc60fbb3481df548'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.359359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56451a43ab34353bc60fbb3481df548'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.361613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56451a43ab34353bc60fbb3481df548'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.374612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56451a43ab34353bc60fbb3481df548'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.375213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b0edced-7dc7-4dda-864d-dda75226df4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56451a43ab34353bc60fbb3481df548'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.437278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ed080a9be74314800fc54d575464bb', 'request_id': 'req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.446265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1029/2038] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:56 2022] DELETE /identity/v3/users/d56451a43ab34353bc60fbb3481df548 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:57.453499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:57.453793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:57.454187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:57.455184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ed080a9be74314800fc54d575464bb', 'request_id': 'req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.502838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ed080a9be74314800fc54d575464bb', 'request_id': 'req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.533665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:57.535617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:57.548893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] The token cache is being invalidate because project a6ed080a9be74314800fc54d575464bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:57.550940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.552703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-13b61fe6-7751-4d62-bd5d-9730a0bcdcb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:57.560893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1010/2039] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:57 2022] DELETE /identity/v3/projects/a6ed080a9be74314800fc54d575464bb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:57.567091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd35b9-ba2c-452d-8346-f9803a86d593 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:57.567600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd35b9-ba2c-452d-8346-f9803a86d593 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:57.568030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd35b9-ba2c-452d-8346-f9803a86d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:57.633666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9dfd35b9-ba2c-452d-8346-f9803a86d593 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:57.634997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9dfd35b9-ba2c-452d-8346-f9803a86d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:57.755950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1030/2040] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:57.773799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:57.776921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:57.918161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1011/2041] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:57.946188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:57.947674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:57.947990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.005286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab1d25-ac73-4cf9-bfd4-87416f9c89b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.016261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.016859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.017173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.017504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.017680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:58.021395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:58.046064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.047252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1012/2042] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:58.047776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.048304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.076495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d5c23-6c75-4451-89d4-41a8b552715d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.090315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.091024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.092365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.092365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.092667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:25:58.093160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.098127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.112135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1031/2043] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:25:58.131405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1013/2044] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:25:58.136766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.169905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7191ed7-1c06-4aa3-8852-e92865fe033d service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.185112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.185929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.186217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.186584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.187350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.189950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.267683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1032/2045] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:58 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:58.271786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.307175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716da5cf-36a1-4ac7-ab0b-311d5e145059 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.427135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:58.459247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.460417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.460921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.462337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.462726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.482736 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.542661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1014/2046] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:58 2022] POST /identity/v3/users => generated 284 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:25:58.546754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.569778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002a7d-6dee-4707-878c-ca4e1dc17f5f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.597364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.598410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.598862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.599331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.600372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.602259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.632219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1033/2047] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:58.637071 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.649060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91251-3ec7-4bbb-a882-8451595ebb69 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.655151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.655151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.655595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.655752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.656553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.658370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a16e37-c71e-4004-b6a0-162b65c5447a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.673920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1015/2048] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:25:58.675846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.694381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb7e2-78ea-42b6-9043-8f4f611c8736 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:58.704910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:58.706910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.706910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.707432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] PATH_INFO: `/v3/projects/5db8ffb6d5be4c488db23c5a6c85753e/users/f9165ff894414e91bb738475d3ef9c50/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.730719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:58.742165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db8ffb6d5be4c488db23c5a6c85753e, user_id=f9165ff894414e91bb738475d3ef9c50, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:58.749904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:58.782192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1034/2049] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:58 2022] PUT /identity/v3/projects/5db8ffb6d5be4c488db23c5a6c85753e/users/f9165ff894414e91bb738475d3ef9c50/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:58.797810 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:58.798129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:58.798323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:58.844697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] MFA Rules not processed for user `f9165ff894414e91bb738475d3ef9c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:25:58.845999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:58.992556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:58.996866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:59.033255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1016/2050] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:25:59.040166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:59.136425 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891f4c-2a38-410d-95a4-b2b8a78b137c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:59.142759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:59.144242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:59.144829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:59.145302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] PATH_INFO: `/v3/projects/0df93b27f35b40a488e3f8c636fde300` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:59.147837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df93b27f35b40a488e3f8c636fde300)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:59.152424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:59.181703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f60c-2da1-4f19-a41f-ddc594c9486d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:59.197590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:59.201270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:59.202128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:59.202510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:59.204550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:59.211505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:59.272904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df93b27f35b40a488e3f8c636fde300', 'request_id': 'req-fb3ae04a-d192-4b49-ad31-6acfe3901290'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.278922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1017/2051] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:25:59 2022] POST /identity/v3/projects => generated 367 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:59.284821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:59.300827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df93b27f35b40a488e3f8c636fde300', 'request_id': 'req-fb3ae04a-d192-4b49-ad31-6acfe3901290'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.332627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df93b27f35b40a488e3f8c636fde300', 'request_id': 'req-fb3ae04a-d192-4b49-ad31-6acfe3901290'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.364260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] The token cache is being invalidate because project 0df93b27f35b40a488e3f8c636fde300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:25:59.365807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.368391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.380981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1035/2052] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:25:58 2022] DELETE /identity/v3/projects/0df93b27f35b40a488e3f8c636fde300 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:25:59.382480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:59.386292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:59.488389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:59.523613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae04a-d192-4b49-ad31-6acfe3901290 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:59.532226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:59.533587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:59.533881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:59.534167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] PATH_INFO: `/v3/users/aec3e6f6d00244f1b1d77c79a35c1cf8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:59.536057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec3e6f6d00244f1b1d77c79a35c1cf8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:59.539178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:59.568289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f4f60-6b54-4411-99f4-c1b62e8646e4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:59.589200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:25:59.598130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:59.604072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:59.604952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:59.610015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:59.615503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:59.622211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:59.690066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1018/2053] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:25:59 2022] POST /identity/v3/users => generated 290 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:25:59.694732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:59.700936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:59.708915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec3e6f6d00244f1b1d77c79a35c1cf8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.732195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e1473-b335-4489-8f76-27c50c09b008 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:25:59.733828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec3e6f6d00244f1b1d77c79a35c1cf8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.734677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec3e6f6d00244f1b1d77c79a35c1cf8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.745751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:25:59.747571 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:25:59.748541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:25:59.749226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:25:59.751101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:25:59.754456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:25:59.756205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec3e6f6d00244f1b1d77c79a35c1cf8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.758287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec3e6f6d00244f1b1d77c79a35c1cf8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:25:59.794480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1019/2054] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:25:59.802207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:25:59.806405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:25:59.825508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1036/2055] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:25:59 2022] DELETE /identity/v3/users/aec3e6f6d00244f1b1d77c79a35c1cf8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:25:59.851560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2cb0-1dc2-4fba-b028-b6a1d7d35049 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:00.055135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:00.056915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.057356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.063097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] PATH_INFO: `/v3/projects/b3aa358a96564132aaa86ce25fbe3806/users/6fff967a9ab84753a8eaa3aebea63685/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.092197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3aa358a96564132aaa86ce25fbe3806, user_id=6fff967a9ab84753a8eaa3aebea63685, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:00.097764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:00.132989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1020/2056] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:25:59 2022] PUT /identity/v3/projects/b3aa358a96564132aaa86ce25fbe3806/users/6fff967a9ab84753a8eaa3aebea63685/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:00.134368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:00.156331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ea87-51da-4dca-8a5e-7dcde13801c2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:00.319022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:00.322999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.323514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.323821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.325692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:00.328081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f69547-b9db-410d-ade2-0c888830767f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:00.347434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1037/2057] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:00.352298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:00.374141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26fc80-123a-4b2a-9d3c-97f3ad9179f9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:00.384823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:00.386639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.387618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.388204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] PATH_INFO: `/v3/projects/b3aa358a96564132aaa86ce25fbe3806/users/6fff967a9ab84753a8eaa3aebea63685/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.394102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3aa358a96564132aaa86ce25fbe3806, user_id=6fff967a9ab84753a8eaa3aebea63685, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:00.400164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:00.422712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1021/2058] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:00 2022] PUT /identity/v3/projects/b3aa358a96564132aaa86ce25fbe3806/users/6fff967a9ab84753a8eaa3aebea63685/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:00.442967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.442967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.443399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.511211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] MFA Rules not processed for user `6fff967a9ab84753a8eaa3aebea63685`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:00.514362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:00.699685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:00.701458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:00.731494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1038/2059] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:00.746241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:00.749042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:00.821839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:00.868852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd4c7-592c-4368-abbb-bf5e242c798a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:00.877712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:00.879843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.880438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.880837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] PATH_INFO: `/v3/users/ebf5eb935fc84b4dbc3090186cae2144` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.882501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf5eb935fc84b4dbc3090186cae2144)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:00.884936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:00.932901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c00a3-1800-4686-b018-e426c4e76384 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:00.940713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:00.942316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:00.942483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:00.943449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:00.961827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:00.966216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:00.990789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:00.996037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1039/2060] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:01.019775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf5eb935fc84b4dbc3090186cae2144'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.045081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf5eb935fc84b4dbc3090186cae2144'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.070646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf5eb935fc84b4dbc3090186cae2144'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.094287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:01.097623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.108260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf5eb935fc84b4dbc3090186cae2144'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.109233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-553f19fa-d96c-4ff8-8d90-319153aee05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf5eb935fc84b4dbc3090186cae2144'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.131427 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1022/2061] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:00 2022] DELETE /identity/v3/users/ebf5eb935fc84b4dbc3090186cae2144 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:01.137236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.138822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918379b8-f9be-412b-b4d1-39cb5ea6f1e4 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:01.139284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.139694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.219314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:01.220986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.250184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:01.251806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.251806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.251806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.255685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.407500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:01.408904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:01.409288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1023/2062] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:01.410488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:01.421260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.426368 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1040/2063] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:01.427571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:01.430224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.447263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c83ac8-3517-4134-abca-8ad90879bbde None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:01.459424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:01.460339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.460629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.461263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] PATH_INFO: `/v3/projects/c54f18fc25434fad8c644fabd7d98aee` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.462724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54f18fc25434fad8c644fabd7d98aee)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:01.464845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:01.574292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f399-6da7-4ed3-8352-08923538d572 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:01.585141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54f18fc25434fad8c644fabd7d98aee', 'request_id': 'req-607f6421-c0fa-4355-987e-2cfafaa598f6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.596567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54f18fc25434fad8c644fabd7d98aee', 'request_id': 'req-607f6421-c0fa-4355-987e-2cfafaa598f6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.612094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54f18fc25434fad8c644fabd7d98aee', 'request_id': 'req-607f6421-c0fa-4355-987e-2cfafaa598f6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.632664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] The token cache is being invalidate because project c54f18fc25434fad8c644fabd7d98aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:01.634246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.635625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-607f6421-c0fa-4355-987e-2cfafaa598f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:01.639679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1024/2064] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:01 2022] DELETE /identity/v3/projects/c54f18fc25434fad8c644fabd7d98aee => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:01.647130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.647356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:01.648781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.649858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.678967 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:01.680786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.694634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:01.695380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.695670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.695959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.699330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:01.812252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:01.814064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08487b-5319-4fd1-b834-f60437f205e5 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:01.829111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:01.850035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1041/2065] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:01.900882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1025/2066] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:01.915861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:01.917320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:01.917902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:01.997607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:02.002656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:02.041941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1042/2067] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:02.044252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.088128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b555b-605b-48a0-8e82-17fe23eb9f3b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.113776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.116033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.119785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.121379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.123190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:02.127583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.131398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.208763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.212864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1026/2068] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:02.236194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564a418-8307-4075-8ad3-00d1dc21ad32 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.246489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.248112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.248868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.254815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.256729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.260794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.341036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.349388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1043/2069] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:02 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:02.358866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a3cc6-378c-451a-ac43-feaef3281a74 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.501553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.503853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.504670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.506521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.508526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.516613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.559591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.572152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1027/2070] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:02 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:02.579898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6c5d-32f3-4104-aa9a-6d17a2b18d90 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.589770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.600006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.600302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.600633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.605473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.611802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.631243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.641323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1044/2071] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:02.644076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407330d-db08-49b2-a9f7-089f7acbd9ef admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.650036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.650924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.651277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.651905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.653120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.655616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4016-ab59-4e0f-a2b0-8d30c036abd6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.679982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.681497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1028/2072] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:02.695442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc356-a7a0-46a6-a16b-c132154f0f22 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:02.704321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:02.705265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.705593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.705972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] PATH_INFO: `/v3/projects/32fc8cf8feb741578e2669ae36b48ee8/users/4fcdab139ef3451cb22776e9282314fa/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.734010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=32fc8cf8feb741578e2669ae36b48ee8, user_id=4fcdab139ef3451cb22776e9282314fa, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:02.737851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:02.770997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1045/2073] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:02 2022] PUT /identity/v3/projects/32fc8cf8feb741578e2669ae36b48ee8/users/4fcdab139ef3451cb22776e9282314fa/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:02.772922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:02.775835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:02.776858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:02.878426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] MFA Rules not processed for user `4fcdab139ef3451cb22776e9282314fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:02.881931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:02.942412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:02.946752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:03.136475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1029/2074] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:03.146785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:03.158845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f91281-4367-4f23-a497-4132c1b94ade admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.169647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca548-bd40-43db-bca2-6624edf7c226 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.172986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.178990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.182180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.182180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.182180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.186755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.189151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.189440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:03.189606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] PATH_INFO: `/v3/projects/ea07dc4bdd584fdc8e8bfb24b7f4655c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.210483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea07dc4bdd584fdc8e8bfb24b7f4655c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:03.211516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:03.223026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:03.315058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:03.384137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea07dc4bdd584fdc8e8bfb24b7f4655c', 'request_id': 'req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.391365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1030/2075] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:03 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:03.394897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:03.400268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea07dc4bdd584fdc8e8bfb24b7f4655c', 'request_id': 'req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.415526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea07dc4bdd584fdc8e8bfb24b7f4655c', 'request_id': 'req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.470090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] The token cache is being invalidate because project ea07dc4bdd584fdc8e8bfb24b7f4655c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:03.472088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.474387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:03.482327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.486050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1046/2076] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:02 2022] DELETE /identity/v3/projects/ea07dc4bdd584fdc8e8bfb24b7f4655c => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:03.491789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:03.495989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:03.689580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4884-35e2-438d-9bdd-4087c6a9826a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.700263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.701935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.702712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.703486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.705310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:03.711730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:03.724114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ff9d8-3d8a-41bb-bdb8-8ae81deaf2ed admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.754943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.756110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.756486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.756812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] PATH_INFO: `/v3/users/493e1b1ac3e04569948d09ead82a20a4` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.762590 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493e1b1ac3e04569948d09ead82a20a4)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:03.766567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:03.812690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1031/2077] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:03 2022] POST /identity/v3/users => generated 298 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:03.814113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:03.817180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:03.846980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefd389-2ef7-46cc-ad4f-b9e61703bbd5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.857282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.859049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.860055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.860748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.862453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:03.866234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:03.907163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493e1b1ac3e04569948d09ead82a20a4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.908726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1032/2078] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:03.916837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:03.935816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b36b73-4530-4992-97b6-337ba60d1a99 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:03.947321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493e1b1ac3e04569948d09ead82a20a4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.950370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493e1b1ac3e04569948d09ead82a20a4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.953101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:03.954920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:03.955295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:03.955649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] PATH_INFO: `/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b/users/ce9e1746a4fd4b0680edfbbbadeee21f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:03.974275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493e1b1ac3e04569948d09ead82a20a4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.979712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493e1b1ac3e04569948d09ead82a20a4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:03.993280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6d6b7a66fc472096c3d3a457b7a26b, user_id=ce9e1746a4fd4b0680edfbbbadeee21f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:04.009004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:04.051529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1033/2079] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:03 2022] PUT /identity/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b/users/ce9e1746a4fd4b0680edfbbbadeee21f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:04.051826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1047/2080] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:03 2022] DELETE /identity/v3/users/493e1b1ac3e04569948d09ead82a20a4 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:04.053920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:04.055744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:04.273933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4820a1-e6ee-4227-bb66-40ed54fdcb33 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:04.497064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:04.498385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:04.499199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:04.499980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:04.501847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:04.503950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a22d-7d5d-4cd5-bf0e-8c9f8d1235d2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:04.551167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:04.563512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1034/2081] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:04.575479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cc0d-5e82-4577-a9ff-b1eec5b30313 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:04.582838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:04.584380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:04.584733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:04.584974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] PATH_INFO: `/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b/users/ce9e1746a4fd4b0680edfbbbadeee21f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:04.588017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6d6b7a66fc472096c3d3a457b7a26b, user_id=ce9e1746a4fd4b0680edfbbbadeee21f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:04.591625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:04.619724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1048/2082] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:04 2022] PUT /identity/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b/users/ce9e1746a4fd4b0680edfbbbadeee21f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:04.624316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:04.629107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:04.630437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:04.755315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:04.782406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] MFA Rules not processed for user `ce9e1746a4fd4b0680edfbbbadeee21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:04.798323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.035653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1035/2083] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:05.049141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:05.052873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.098883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:05.129593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:05.136129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.242747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed551ed-3f6d-4e9c-b840-b615a75b7d6b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:05.251186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:05.251900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:05.252188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:05.252465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:05.255367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.323465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24193ca7-a248-4ce0-88b8-a298f4f8861b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:05.335083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:05.337043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:05.337634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:05.338165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] PATH_INFO: `/v3/projects/2c55d64432254c25bc1cc5c438bf7db2` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:05.343021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c55d64432254c25bc1cc5c438bf7db2)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:05.354097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:05.378829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:05.381689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e518328-b66d-46e6-9cde-7c9e08b1c6d7 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:05.407290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1049/2084] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:05.550031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c55d64432254c25bc1cc5c438bf7db2', 'request_id': 'req-125b3349-375e-420f-a7ea-c4261a5b9127'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:05.571551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c55d64432254c25bc1cc5c438bf7db2', 'request_id': 'req-125b3349-375e-420f-a7ea-c4261a5b9127'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:05.606933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c55d64432254c25bc1cc5c438bf7db2', 'request_id': 'req-125b3349-375e-420f-a7ea-c4261a5b9127'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:05.633005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] The token cache is being invalidate because project 2c55d64432254c25bc1cc5c438bf7db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:05.633659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:05.634950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-125b3349-375e-420f-a7ea-c4261a5b9127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:05.638824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1036/2085] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:05 2022] DELETE /identity/v3/projects/2c55d64432254c25bc1cc5c438bf7db2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:05.645902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:05.646882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:05.647818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:05.699638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:05.702477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.885124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1050/2086] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:05.896202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5551d-c440-4ca7-8e08-e40a3dbb766b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:05.896677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5551d-c440-4ca7-8e08-e40a3dbb766b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:05.897468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5551d-c440-4ca7-8e08-e40a3dbb766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:05.946326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f2e5551d-c440-4ca7-8e08-e40a3dbb766b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:05.948161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f2e5551d-c440-4ca7-8e08-e40a3dbb766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:05.976215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1037/2087] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:05.982400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:05.998444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf64bf-4ac6-4309-a740-1c1051c17a72 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.025094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.038704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.039586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.040225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.042984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:06.043610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.048333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013081e0-e3e1-4d44-b086-5efa1a7d51eb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.076168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017fa15-e622-4778-9983-a4dd59c1dbcf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.076693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017fa15-e622-4778-9983-a4dd59c1dbcf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.077523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017fa15-e622-4778-9983-a4dd59c1dbcf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.079668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1038/2088] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:26:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 16:26:06.105738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1051/2089] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:06.107326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017fa15-e622-4778-9983-a4dd59c1dbcf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:06.143168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017fa15-e622-4778-9983-a4dd59c1dbcf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.178213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.179392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.179838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.180388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.181834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.185866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.186455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.188740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.189842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.245978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:06.262328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:06.265688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:06.276336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1039/2090] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:06 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:06.278902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:06.459223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:06.532233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1052/2091] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:26:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:06.535394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:06.542944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:06.608263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e6b21-673c-4a1a-93d9-c5b3e8723302 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.617770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.618917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.619614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.620100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.621485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.625709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.688960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1040/2092] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:06 2022] POST /identity/v3/users => generated 293 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:06.688960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:06.697233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:06.752678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180e2ed-13d5-4e3e-9083-86c71e49d6d5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.775473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.776635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.777095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.778068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.778914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cafb40-da00-48e0-af82-91614f7cede4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.791048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.792809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.793311 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.808962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] PATH_INFO: `/v3/users/f2d0f7e3f1cc4955a0ca99d588344322` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.812363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d0f7e3f1cc4955a0ca99d588344322)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.815691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.816513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.821391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.848866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1041/2093] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:06.851178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:06.901254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885d05-32e5-4d6a-b387-80ad31247a4c service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:06.935824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:06.941922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:06.942499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:06.942952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:06.944603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:06.952973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:06.969173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d0f7e3f1cc4955a0ca99d588344322'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:06.992211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1042/2094] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:06.995182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:07.005701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d0f7e3f1cc4955a0ca99d588344322'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:07.010775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d0f7e3f1cc4955a0ca99d588344322'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:07.021657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808ace9-c34c-49bd-ba4e-7725187b1e5b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:07.030201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d0f7e3f1cc4955a0ca99d588344322'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:07.037774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7aa0115e-8e22-4797-ae4a-d443f6e91319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d0f7e3f1cc4955a0ca99d588344322'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:07.045042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:07.046913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:07.047373 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:07.047660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:07.050502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:07.055892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:07.089552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1043/2095] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:07.095471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:07.103011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1053/2096] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:06 2022] DELETE /identity/v3/users/f2d0f7e3f1cc4955a0ca99d588344322 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:07.108885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:07.136942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500101-a7e6-43df-b762-2ed6147c9586 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:07.330317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:07.332669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:07.332938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:07.333634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] PATH_INFO: `/v3/projects/c277be7923c2427c860030ab31a64b96/users/a9f91915b7e54295b6dec28327db576c/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:07.356888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c277be7923c2427c860030ab31a64b96, user_id=a9f91915b7e54295b6dec28327db576c, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:07.360506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:07.385555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1044/2097] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:07 2022] PUT /identity/v3/projects/c277be7923c2427c860030ab31a64b96/users/a9f91915b7e54295b6dec28327db576c/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:07.408719 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:07.409395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:07.409807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:07.510008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] MFA Rules not processed for user `a9f91915b7e54295b6dec28327db576c`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:07.512771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:07.642954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:07.645517 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:07.753911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:07.756249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1054/2098] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:07.762575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:07.773793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:07.837208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa7b2-626b-445e-8487-3adc46f95124 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:07.856754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:07.858285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:07.858797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:07.869453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] PATH_INFO: `/v3/projects/287b6d7980174314b20fe6b172725bf4` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:07.872416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=287b6d7980174314b20fe6b172725bf4)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:07.876094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.026291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287b6d7980174314b20fe6b172725bf4', 'request_id': 'req-517cc6de-318e-422d-b27a-bbd951a5db08'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:08.045422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287b6d7980174314b20fe6b172725bf4', 'request_id': 'req-517cc6de-318e-422d-b27a-bbd951a5db08'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:08.052884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452d944-6493-41cd-b9de-acf369ea1675 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.074983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287b6d7980174314b20fe6b172725bf4', 'request_id': 'req-517cc6de-318e-422d-b27a-bbd951a5db08'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:08.104173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] The token cache is being invalidate because project 287b6d7980174314b20fe6b172725bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:08.104356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:08.105393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-517cc6de-318e-422d-b27a-bbd951a5db08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:08.108391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1045/2099] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:07 2022] DELETE /identity/v3/projects/287b6d7980174314b20fe6b172725bf4 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:08.120006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f75c7-744f-48b8-9a22-f80085c65c46 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.120529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f75c7-744f-48b8-9a22-f80085c65c46 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.121023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f75c7-744f-48b8-9a22-f80085c65c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.199794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.200881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.201383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.201867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.203289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.206845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.209290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1d0f75c7-744f-48b8-9a22-f80085c65c46 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:08.217127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1d0f75c7-744f-48b8-9a22-f80085c65c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:08.242966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1046/2100] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:08.260912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.262038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.262658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.281736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1055/2101] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:07 2022] POST /identity/v3/projects => generated 385 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:08.290329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.296051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:08.312225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:08.314925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:08.440256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1047/2102] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:08.446058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.460385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc3db5-606a-440f-8bf0-d3d8ade7f63c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.463656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94521-a563-4acf-a47c-db70a59170bd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.470687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.471566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.471860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.472143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.474152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:08.474694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.477471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.487254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.488597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.489176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.489728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.491357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.499362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.499961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1048/2103] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:26:08.500760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.515898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9e62-d57c-4bb2-a7fe-4e6ba709b798 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.524306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.527546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.527874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.528369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.531690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.537909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.562664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1056/2104] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:08 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:08.568325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.591147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1049/2105] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:08 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:08.593698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.620093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aa86e-4bdd-4671-a894-03830e76ab58 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.701240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:08.732516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:08.790773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a74d8-7c5f-49d3-8c45-b8c128926699 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.798392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.799348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.799766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.800369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.803138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.806055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.855017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1050/2106] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:08 2022] POST /identity/v3/users => generated 303 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:08.862020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.888784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.892277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.897933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.897933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.898716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.898891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb528c9-69f5-4722-8ea4-30806868cef1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.902139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.917262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.918477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.918767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.919076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.919938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.923331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.932198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1057/2107] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:08.937361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.955293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1051/2108] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:08.956310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:08.972504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4643e-ab2f-4139-80fb-37e57d463006 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:08.985491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:08.986567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:08.986957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:08.987616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:08.988745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:08.991695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:08.995535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d696c-a9b6-415a-a75d-6fab0c73629d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.011257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1058/2109] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:09.012918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.015435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.015919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.016163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.016452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.016734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.017531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.019872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.036353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ab1b6-f7ed-4a9d-8d1c-85ffaa65a24b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.048040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1052/2110] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:09.049037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.053408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.054589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.054877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.055305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] PATH_INFO: `/v3/users/f9165ff894414e91bb738475d3ef9c50` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.061291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9165ff894414e91bb738475d3ef9c50)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.067825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.068144 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd332c1-a150-4f2d-9c75-18fb9b0591bd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.074852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.076378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.076741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.077219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] PATH_INFO: `/v3/projects/41affccca2c84fc8ba2d494f6cafe1fa/users/7f564d9a814d493a88461f433c4dc317/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.110180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41affccca2c84fc8ba2d494f6cafe1fa, user_id=7f564d9a814d493a88461f433c4dc317, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.116870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.153889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1053/2111] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:09 2022] PUT /identity/v3/projects/41affccca2c84fc8ba2d494f6cafe1fa/users/7f564d9a814d493a88461f433c4dc317/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:09.154606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.197780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9165ff894414e91bb738475d3ef9c50'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:09.208341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9165ff894414e91bb738475d3ef9c50'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:09.209225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9165ff894414e91bb738475d3ef9c50'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:09.221244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9165ff894414e91bb738475d3ef9c50'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:09.222413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f4c5463a-f509-4351-bac9-165ccc7b58e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9165ff894414e91bb738475d3ef9c50'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:09.248495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1059/2112] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:09 2022] DELETE /identity/v3/users/f9165ff894414e91bb738475d3ef9c50 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:09.250235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955244a6-93b6-4313-bda7-bf078c234633 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.250533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c55d1-4a50-484f-8675-07e444d85443 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.250848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955244a6-93b6-4313-bda7-bf078c234633 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.251183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955244a6-93b6-4313-bda7-bf078c234633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.293151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-955244a6-93b6-4313-bda7-bf078c234633 None None] MFA Rules not processed for user `7f564d9a814d493a88461f433c4dc317`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:09.294659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-955244a6-93b6-4313-bda7-bf078c234633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.371898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.373542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.374226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.374599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] PATH_INFO: `/v3/projects/70e2d6663fe14da1bdff6f1e6e247a2b/users/affa1d1992bd4c2b8b7166ae3081b23d/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.407413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e2d6663fe14da1bdff6f1e6e247a2b, user_id=affa1d1992bd4c2b8b7166ae3081b23d, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.413176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.416292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1060/2113] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:09.420173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09c2d-6523-4740-ac51-7e30e9483b71 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.421224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09c2d-6523-4740-ac51-7e30e9483b71 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.421768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09c2d-6523-4740-ac51-7e30e9483b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.428717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:09.445188 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1054/2114] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:09 2022] PUT /identity/v3/projects/70e2d6663fe14da1bdff6f1e6e247a2b/users/affa1d1992bd4c2b8b7166ae3081b23d/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:09.451020 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.451663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.459182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cce09c2d-6523-4740-ac51-7e30e9483b71 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:09.461857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cce09c2d-6523-4740-ac51-7e30e9483b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.478961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:09.564034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a67f-6868-4d28-a2fa-0822bf219280 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.570982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.571812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.572239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.572651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.573718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.576092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.595108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1061/2115] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:09.597946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.598503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.599478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.622196 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1055/2116] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:09 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:09.624379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.628063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.634965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:09.637202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.723238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1062/2117] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:09.724428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.737763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a231ebc-0b8e-4ba4-9cdd-536994446c83 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.745761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.746919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.747511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.748031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.748965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.751681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.757506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df6425-c200-4f39-9ffa-b9932edca53e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.766881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.769992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.769992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.769992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.789637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1063/2118] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:09 2022] POST /identity/v3/users => generated 299 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:09.789637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.790111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.791610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.794147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:09.811323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1056/2119] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:09.812875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.818485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a155898-64f4-45ab-8f68-87c571d985aa admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.825656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.827030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.827512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.828093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.830909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896f1a-e928-45f2-96dd-b38739a06c3f service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.839521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.841016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.841755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.842450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.844189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.847550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.849450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.854482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.868585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1057/2120] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:09.874432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:09.879569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1064/2121] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:09.889824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9edc4a-5470-4384-bee8-b7682c282620 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:09.899280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:09.900927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:09.901454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:09.902134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] PATH_INFO: `/v3/projects/98599a877a51432f91cd9ea024a52407/users/714dae18b6c441b4b31dabecc7058859/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:09.937820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=98599a877a51432f91cd9ea024a52407, user_id=714dae18b6c441b4b31dabecc7058859, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:09.943640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:09.989138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:10.000835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1058/2122] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:09 2022] PUT /identity/v3/projects/98599a877a51432f91cd9ea024a52407/users/714dae18b6c441b4b31dabecc7058859/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:10.017562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d4a30-8e29-4450-96dc-192e28255fec service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:10.282412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:10.283557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:10.284042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:10.284508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:10.290292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:10.294422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ab249-a6f5-4df4-88f0-b3ba6577312b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:10.351987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1065/2123] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:10.361924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:10.391021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b0bfc-bbe3-4740-a8f9-557067ed5fff admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:10.408536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:10.411856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:10.412472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:10.412865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] PATH_INFO: `/v3/projects/98599a877a51432f91cd9ea024a52407/users/714dae18b6c441b4b31dabecc7058859/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:10.438379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98599a877a51432f91cd9ea024a52407, user_id=714dae18b6c441b4b31dabecc7058859, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:10.449881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:10.539231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1059/2124] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:10 2022] PUT /identity/v3/projects/98599a877a51432f91cd9ea024a52407/users/714dae18b6c441b4b31dabecc7058859/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:10.544896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:10.545535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:10.546006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:10.670759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] MFA Rules not processed for user `714dae18b6c441b4b31dabecc7058859`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:10.675284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:10.708459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:10.984301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1066/2125] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:11.023841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:11.029169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:11.138741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:11.168871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273cc2-03a8-4bae-b835-52b1872cb5d8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:11.178518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:11.179961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:11.180894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:11.181378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:11.185100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:11.287780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:11.290352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:11.335639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:11.339558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2e460-686d-405e-aca0-7f4caba1df2b service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:11.360022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1060/2126] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:11.433440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebb829-edc5-4c65-a64b-1aa00cf62a94 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:11.451318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:11.455412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:11.456026 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:11.456372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] PATH_INFO: `/v3/projects/5db8ffb6d5be4c488db23c5a6c85753e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:11.457938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db8ffb6d5be4c488db23c5a6c85753e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:11.460093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:11.538210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:11.538578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:11.538920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:11.583052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db8ffb6d5be4c488db23c5a6c85753e', 'request_id': 'req-523c9ff2-3486-47b7-97a3-62077d74f5be'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:11.589640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] MFA Rules not processed for user `affa1d1992bd4c2b8b7166ae3081b23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:11.591955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:11.612013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db8ffb6d5be4c488db23c5a6c85753e', 'request_id': 'req-523c9ff2-3486-47b7-97a3-62077d74f5be'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:11.628385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db8ffb6d5be4c488db23c5a6c85753e', 'request_id': 'req-523c9ff2-3486-47b7-97a3-62077d74f5be'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:11.660881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] The token cache is being invalidate because project 5db8ffb6d5be4c488db23c5a6c85753e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:11.662823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:11.664877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:11.668162 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1067/2127] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:11 2022] DELETE /identity/v3/projects/5db8ffb6d5be4c488db23c5a6c85753e => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:11.679215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:11.687141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:11.824966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:11.845615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1061/2128] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:11 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:11.857524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:11.862496 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:12.048945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9ff2-3486-47b7-97a3-62077d74f5be admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.087665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.092396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.092988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.095147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] PATH_INFO: `/v3/users/6fff967a9ab84753a8eaa3aebea63685` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.100489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fff967a9ab84753a8eaa3aebea63685)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.112610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.276079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55648e-a08b-4b0f-9685-502fccb2122c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.301851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.303656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.304291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.304653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.311392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.341445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.352823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fff967a9ab84753a8eaa3aebea63685'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:12.381976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fff967a9ab84753a8eaa3aebea63685'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:12.384198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fff967a9ab84753a8eaa3aebea63685'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:12.401518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fff967a9ab84753a8eaa3aebea63685'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:12.406011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fff967a9ab84753a8eaa3aebea63685'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:12.412768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1062/2129] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:11 2022] POST /identity/v3/projects => generated 405 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:12.414702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:12.418035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:12.440285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1068/2130] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:11 2022] DELETE /identity/v3/users/6fff967a9ab84753a8eaa3aebea63685 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:12.441947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:12.444323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:12.444600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:12.527776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac881f-501d-4ed0-a210-eb2884b9bf62 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.595701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7ded-17cb-4655-b312-5e32b60242bc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.601795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.602693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.603124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.603404 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.604512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.606793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.639992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1069/2131] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:12 2022] POST /identity/v3/users => generated 309 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:12.655662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:12.685141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ef8b-af12-4b70-acaa-ec96320cd20c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.704354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.708907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.709383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.709911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.710597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.711435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.711819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.712418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.712755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.715115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:12.716652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.746972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1070/2132] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:12.750840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:12.780667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cd3b9-c4b2-40b1-82ca-b07586bfae7f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:12.792497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:12.793936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:12.794309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:12.794690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] PATH_INFO: `/v3/projects/e7453c1f59784041996c3372c732d899/users/8f0da915bdbf4a32a4c91d689def3182/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:12.827555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7453c1f59784041996c3372c732d899, user_id=8f0da915bdbf4a32a4c91d689def3182, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.831055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.856401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1071/2133] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:12 2022] PUT /identity/v3/projects/e7453c1f59784041996c3372c732d899/users/8f0da915bdbf4a32a4c91d689def3182/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:12.856951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:12.858784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:12.862159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:12.872885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1063/2134] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:12.877303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:13.031347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ee2a-6f50-498c-ba0b-7dece599fe6c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:13.043164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:13.044436 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.044869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.045328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.046540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:13.049104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:13.076787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1072/2135] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:13.078084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:13.107354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:13.112711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:13.118337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d0145-3c36-4a5d-8a56-15ccb7f52b77 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:13.149368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:13.151349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.152181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.154536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] PATH_INFO: `/v3/projects/e7453c1f59784041996c3372c732d899/users/8f0da915bdbf4a32a4c91d689def3182/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.155158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0236a3c-6834-4971-a785-06ceacc1bd11 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:13.164695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7453c1f59784041996c3372c732d899, user_id=8f0da915bdbf4a32a4c91d689def3182, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:13.168421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:13.170324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.170613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.170887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] PATH_INFO: `/v3/projects/b3aa358a96564132aaa86ce25fbe3806` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.173054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcab2b-00e8-466e-be6c-7c1decbe387f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:13.180131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3aa358a96564132aaa86ce25fbe3806)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:13.185525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:13.196874 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1064/2136] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:13 2022] PUT /identity/v3/projects/e7453c1f59784041996c3372c732d899/users/8f0da915bdbf4a32a4c91d689def3182/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:13.351853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3aa358a96564132aaa86ce25fbe3806', 'request_id': 'req-683369a2-c544-4abf-8727-5dc33e6d06b4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:13.364071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3aa358a96564132aaa86ce25fbe3806', 'request_id': 'req-683369a2-c544-4abf-8727-5dc33e6d06b4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:13.394183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3aa358a96564132aaa86ce25fbe3806', 'request_id': 'req-683369a2-c544-4abf-8727-5dc33e6d06b4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:13.427682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] The token cache is being invalidate because project b3aa358a96564132aaa86ce25fbe3806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:13.428128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:13.429750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-683369a2-c544-4abf-8727-5dc33e6d06b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:13.445759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1073/2137] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:13 2022] DELETE /identity/v3/projects/b3aa358a96564132aaa86ce25fbe3806 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:13.454881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.455750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.456060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.557540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:13.560406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:13.852204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1065/2138] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:13.863157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.864097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.864496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.887692 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:13.890623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:13.899450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:13.909622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:13.914205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:13.937601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1074/2139] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:13.942156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:13.961792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef63a5-ca42-4c9a-99fe-bf0d6dcc8b5e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:13.972488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:13.973205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:13.973523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:13.973804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:13.975222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:13.975555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:13.982840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.005501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1075/2140] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:26:14.022294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:14.056146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5d9b-10cf-48a4-b9ca-88e40e66e769 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.072537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.074881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.075489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.076522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.078091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.084027 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.100752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3054b09-2f1a-4490-b8ce-21fda0605b7a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.108988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.117503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.117939 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.118654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.124910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:14.169106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1076/2141] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:14 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:14.175048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:14.325259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.328819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.335835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:14.362002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1066/2142] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:14.395614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1178-236e-4e74-bc75-e65a8f223133 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.416503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.417992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.418572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.419084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.423239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.440009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b9ae3-ea7e-4f2a-bfbf-850186f950b1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.585431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1077/2143] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:14 2022] POST /identity/v3/users => generated 283 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:14.587015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:14.623280 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388d05-da7d-4385-8bcb-c9651cfa28cb service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.640926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.642692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.643870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.644338 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.645831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.650533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.651024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.653814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.663172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.709648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1067/2144] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:14.731639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:14.754657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624b8c3-861e-4a7d-a7e9-d1e65d9d1104 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.766174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.767945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.769273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.769788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.771379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.775561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.798869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] MFA Rules not processed for user `8f0da915bdbf4a32a4c91d689def3182`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:14.800886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:14.806523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1068/2145] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:14.816400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:14.858715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46648a6-c314-4676-b6f2-da504e33e34f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:14.873153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:14.875187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.885203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.886262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] PATH_INFO: `/v3/projects/62fd23a9b75c45398b6c73c46f80d96d/users/c341cb341b6048458d16665a318da37b/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:14.930999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62fd23a9b75c45398b6c73c46f80d96d, user_id=c341cb341b6048458d16665a318da37b, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:14.935259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f75e0-d658-4bba-bc99-37f9a54fee07 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:14.961724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1069/2146] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:14 2022] PUT /identity/v3/projects/62fd23a9b75c45398b6c73c46f80d96d/users/c341cb341b6048458d16665a318da37b/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:14.982515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:14.984929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:14.985755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:15.081717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] MFA Rules not processed for user `c341cb341b6048458d16665a318da37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:15.081717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:15.098704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1078/2147] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:15.329451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:15.355210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:15.356605 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1070/2148] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:15.378655 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171506c7-cbe9-436a-a033-a55b33289e5f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:15.663716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:15.669928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:15.670482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:15.670914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:15.671748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:15.674524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:15.809412 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:15.827461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1079/2149] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:15 2022] POST /identity/v3/projects => generated 365 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:15.842985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8d495-89c8-45b8-bb2b-6f5a6ae2f9a5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.116703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.118425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.118903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.119474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.120998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.126340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.190061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.200177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1071/2150] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:15 2022] POST /identity/v3/users => generated 289 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:16.228629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159b60-5b14-4404-9bc0-d8abd24aeed9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.255860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.256712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.257043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.258033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.259180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.267589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.300848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1080/2151] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:16.306424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.334452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.340662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22aa-bbd7-46d2-9454-2fe871ecb3cf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.342923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:16.354984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.357240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.358135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.360500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] PATH_INFO: `/v3/projects/7406d087e15b44c79f95fa43baa9689b/users/79440e3d116f43b29f0a2da5a735ddfe/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.425516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7406d087e15b44c79f95fa43baa9689b, user_id=79440e3d116f43b29f0a2da5a735ddfe, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.433050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:16.457418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.516552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db1ec6-f2ca-4bcc-9e15-3fb0711eb5a2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.520779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1072/2152] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:16 2022] PUT /identity/v3/projects/7406d087e15b44c79f95fa43baa9689b/users/79440e3d116f43b29f0a2da5a735ddfe/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:16.525696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.691340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.692513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.693040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.693547 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.697225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:16.728069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69dd22-a3a2-45c0-af0e-64f99f014761 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.739290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.740986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.740986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.741271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.742406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.744683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.767065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1073/2153] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:16.768217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.780942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dbde6-66e7-452f-96c8-629cab6aaf2c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:16.787369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:16.788554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.788847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.789125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] PATH_INFO: `/v3/projects/7406d087e15b44c79f95fa43baa9689b/users/79440e3d116f43b29f0a2da5a735ddfe/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.791579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7406d087e15b44c79f95fa43baa9689b, user_id=79440e3d116f43b29f0a2da5a735ddfe, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.795000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059b3e4-6b4b-4754-b0b5-8fb1e87c4686 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.821273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1074/2154] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:16 2022] PUT /identity/v3/projects/7406d087e15b44c79f95fa43baa9689b/users/79440e3d116f43b29f0a2da5a735ddfe/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:16.823393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04ea3-9cb8-4267-bcd0-4f350d7a85f7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:16.823702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04ea3-9cb8-4267-bcd0-4f350d7a85f7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:16.823989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04ea3-9cb8-4267-bcd0-4f350d7a85f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:16.825626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:16.828107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:16.849230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1081/2155] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:16.873834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:16.876279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:16.877009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-29d04ea3-9cb8-4267-bcd0-4f350d7a85f7 None None] MFA Rules not processed for user `79440e3d116f43b29f0a2da5a735ddfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:16.879425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-29d04ea3-9cb8-4267-bcd0-4f350d7a85f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:16.924609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db201f9e-1273-41f7-ae4a-11574d096a12 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:17.034419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:17.158871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1075/2156] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:17.169513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fc97-6881-4096-b935-689da4355aaa None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.169931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fc97-6881-4096-b935-689da4355aaa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.170482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494fc97-6881-4096-b935-689da4355aaa None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.175808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1076/2157] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:26:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:26:17.188826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494fc97-6881-4096-b935-689da4355aaa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:17.189065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:17.190033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.190439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.191021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] PATH_INFO: `/v3/users/1e3b6b4d699b4038b281ae3ae2b10163` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.194394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3b6b4d699b4038b281ae3ae2b10163)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:17.198976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:17.347566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:17.356474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3b6b4d699b4038b281ae3ae2b10163'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:17.375178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3b6b4d699b4038b281ae3ae2b10163'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:17.376217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3b6b4d699b4038b281ae3ae2b10163'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:17.386334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494fc97-6881-4096-b935-689da4355aaa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:17.388634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3b6b4d699b4038b281ae3ae2b10163'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:17.390934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01771df7-3074-4065-b571-4efd7d379831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3b6b4d699b4038b281ae3ae2b10163'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:17.395370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:17.396966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.397606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.398303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.402557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.430689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:17.432642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1082/2158] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:16 2022] DELETE /identity/v3/users/1e3b6b4d699b4038b281ae3ae2b10163 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:17.434132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1970c3-2058-415c-8024-b7370d3c11bf service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:17.435998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.436861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.437675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.473119 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] MFA Rules not processed for user `ecf1c53cb8784f5c9a2d8d2d71751fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:17.475945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.488110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1077/2159] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:17.494324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.494798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.495092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.573546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:17.587354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.730724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1083/2160] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:17.735732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:17.738295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.904566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1078/2161] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:17.917244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:17.922968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.954554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b09703-76dd-4cc3-9a2a-7ed35a515d05 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:17.964234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:17.965087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:17.965578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:17.966063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:17.972261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:17.997618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd53c4e-9963-487a-a5c8-2d684e786fbb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:18.011940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:18.012944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:18.013632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:18.014390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:18.044966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:18.047796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:18.077683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1079/2162] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:18.079704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:18.092484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:18.106110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:18.111354 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6179e-7df8-4f7f-bafa-a3aa1c6b2619 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:18.146588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1084/2163] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:18.297036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260a338-525f-47b3-80aa-c39410435ed7 service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:18.305788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:18.306937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:18.308524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:18.309242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:18.329614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:18.331295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:18.346257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1080/2164] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:19.017923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:19.019671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:19.020457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:19.099509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:19.102036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:19.107365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] MFA Rules not processed for user `bb3927ae30c54149974d3037c1022854`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:19.110133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:19.124222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:19.153001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd151786-1556-4a09-90a0-61488fca5f87 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:19.171979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:19.173509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:19.173997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:19.174471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] PATH_INFO: `/v3/projects/40df514be5104c5d868870c969214dc5` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:19.181727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=40df514be5104c5d868870c969214dc5)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:19.185857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:19.189061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:19.312608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1085/2165] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:19.378738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40df514be5104c5d868870c969214dc5', 'request_id': 'req-9709e254-a13f-4406-ac40-07e073ab74ed'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.397300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40df514be5104c5d868870c969214dc5', 'request_id': 'req-9709e254-a13f-4406-ac40-07e073ab74ed'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.419147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40df514be5104c5d868870c969214dc5', 'request_id': 'req-9709e254-a13f-4406-ac40-07e073ab74ed'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.476014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] The token cache is being invalidate because project 40df514be5104c5d868870c969214dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:19.482109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.487680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.493816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1081/2166] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:19 2022] DELETE /identity/v3/projects/40df514be5104c5d868870c969214dc5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:19.501431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:19.541535 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:19.544780 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:19.554812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08c615-88eb-435f-b1c7-b78ae2c1098c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:19.690734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:19.692834 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:19.693476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:19.693983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] PATH_INFO: `/v3/users/d0698b2a215447a7aaa637c2264ee8fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:19.696548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0698b2a215447a7aaa637c2264ee8fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:19.700277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:19.822303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0698b2a215447a7aaa637c2264ee8fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.853412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0698b2a215447a7aaa637c2264ee8fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.856372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0698b2a215447a7aaa637c2264ee8fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.859343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e254-a13f-4406-ac40-07e073ab74ed admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:19.874233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:19.877187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:19.877187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:19.878118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] PATH_INFO: `/v3/users/affa1d1992bd4c2b8b7166ae3081b23d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:19.880478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0698b2a215447a7aaa637c2264ee8fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.882618 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=affa1d1992bd4c2b8b7166ae3081b23d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:19.883792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0698b2a215447a7aaa637c2264ee8fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:19.887606 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:19.960224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1086/2167] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:19 2022] DELETE /identity/v3/users/d0698b2a215447a7aaa637c2264ee8fc => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:20.037093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'affa1d1992bd4c2b8b7166ae3081b23d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:20.063267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'affa1d1992bd4c2b8b7166ae3081b23d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:20.078034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'affa1d1992bd4c2b8b7166ae3081b23d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:20.133691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'affa1d1992bd4c2b8b7166ae3081b23d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:20.136113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'affa1d1992bd4c2b8b7166ae3081b23d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:20.150590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1082/2168] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:19 2022] DELETE /identity/v3/users/affa1d1992bd4c2b8b7166ae3081b23d => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:20.798807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:20.801543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:20.881735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:20.945934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906953-5042-46d7-b670-6d933c82abeb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:20.981245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:20.984130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:21.020653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:21.145069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:21.147428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:21.148295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:21.148805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] PATH_INFO: `/v3/users/4fcdab139ef3451cb22776e9282314fa` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:21.154193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcdab139ef3451cb22776e9282314fa)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:21.162086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:21.256717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c51ff-a18f-4775-9d8d-611967c54e5c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:21.280601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:21.282631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:21.290145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:21.291773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] PATH_INFO: `/v3/projects/70e2d6663fe14da1bdff6f1e6e247a2b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:21.295393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e2d6663fe14da1bdff6f1e6e247a2b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:21.298102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:21.349663 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcdab139ef3451cb22776e9282314fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.369728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcdab139ef3451cb22776e9282314fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.371594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcdab139ef3451cb22776e9282314fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.381315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e2d6663fe14da1bdff6f1e6e247a2b', 'request_id': 'req-a930546a-cd45-4b84-815c-ee43b0f87d6a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.392789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcdab139ef3451cb22776e9282314fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.393910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcdab139ef3451cb22776e9282314fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.396228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e2d6663fe14da1bdff6f1e6e247a2b', 'request_id': 'req-a930546a-cd45-4b84-815c-ee43b0f87d6a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.409777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e2d6663fe14da1bdff6f1e6e247a2b', 'request_id': 'req-a930546a-cd45-4b84-815c-ee43b0f87d6a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.439424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] The token cache is being invalidate because project 70e2d6663fe14da1bdff6f1e6e247a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:21.440474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.442155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.449471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1083/2169] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:20 2022] DELETE /identity/v3/projects/70e2d6663fe14da1bdff6f1e6e247a2b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:21.453477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:21.455863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:21.504923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1087/2170] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:20 2022] DELETE /identity/v3/users/4fcdab139ef3451cb22776e9282314fa => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:21.506089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:21.509080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:21.592932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930546a-cd45-4b84-815c-ee43b0f87d6a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:21.634759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742f9b5-2bb1-4cb8-a902-6333b98a8f9a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:21.642239 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:21.644081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:21.644415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:21.644791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] PATH_INFO: `/v3/users/8f0da915bdbf4a32a4c91d689def3182` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:21.646517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0da915bdbf4a32a4c91d689def3182)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:21.648964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:21.651616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:21.702347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:21.732126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:21.733812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:21.734319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:21.734827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] PATH_INFO: `/v3/projects/cf44e760f7144d248c114b04c8a89841` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:21.737250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf44e760f7144d248c114b04c8a89841)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:21.741437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:21.747621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0da915bdbf4a32a4c91d689def3182'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.758879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0da915bdbf4a32a4c91d689def3182'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.760202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0da915bdbf4a32a4c91d689def3182'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.773042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0da915bdbf4a32a4c91d689def3182'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.774358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-84327296-11ca-48bb-bccb-fa48890438bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0da915bdbf4a32a4c91d689def3182'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.810977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1088/2171] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:21 2022] DELETE /identity/v3/users/8f0da915bdbf4a32a4c91d689def3182 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:21.814640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:21.814940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:21.815264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:21.847604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:21.849976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:21.895695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf44e760f7144d248c114b04c8a89841', 'request_id': 'req-e830be6b-7de5-4cdc-8eed-2163db44cd64'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.926245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf44e760f7144d248c114b04c8a89841', 'request_id': 'req-e830be6b-7de5-4cdc-8eed-2163db44cd64'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.954268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf44e760f7144d248c114b04c8a89841', 'request_id': 'req-e830be6b-7de5-4cdc-8eed-2163db44cd64'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:21.977942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1089/2172] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:21.997715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.009767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.024740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] The token cache is being invalidate because project cf44e760f7144d248c114b04c8a89841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:22.025567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.027166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e830be6b-7de5-4cdc-8eed-2163db44cd64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.039043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1084/2173] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:21 2022] DELETE /identity/v3/projects/cf44e760f7144d248c114b04c8a89841 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:22.048290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28e33-ada1-4a7f-b4bc-72daba31a331 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.050524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28e33-ada1-4a7f-b4bc-72daba31a331 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.051777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28e33-ada1-4a7f-b4bc-72daba31a331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.100840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3bb28e33-ada1-4a7f-b4bc-72daba31a331 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:22.112584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3bb28e33-ada1-4a7f-b4bc-72daba31a331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.133996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1085/2174] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:22.152407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.152800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.153252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.199475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bc4fb-59cd-4c23-a9cb-1796f02151c0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:22.207505 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:22.209241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:22.210633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.211140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.211628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.213457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.225720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.228034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1086/2175] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:22.234058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.261189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c2bd0-83bf-4de2-b53b-b8c2c4816932 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:22.262451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:22.276242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:22.287266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:22.288561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.288842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.289244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.292279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:22.292929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:22.298875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:22.322994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1090/2176] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:22.331356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.334946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.351194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1087/2177] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:22.353090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.384509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dd34c-72e5-4654-87a2-f4299c2b6cfe service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:22.394958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:22.396826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.397599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.398767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] PATH_INFO: `/v3/users/a9f91915b7e54295b6dec28327db576c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.401588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f91915b7e54295b6dec28327db576c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:22.405960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:22.416731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9585-c83e-4d8f-82ad-f7ac3dcc4398 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:22.454886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:22.454886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.457494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:22.457673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.457673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.463178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:22.479024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:22.601538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1088/2178] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:22 2022] POST /identity/v3/projects => generated 409 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:22.601538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.609805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.620057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f91915b7e54295b6dec28327db576c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.642205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f91915b7e54295b6dec28327db576c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.643934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f91915b7e54295b6dec28327db576c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.685200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f91915b7e54295b6dec28327db576c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.686561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f91915b7e54295b6dec28327db576c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:22.774846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3e83-9bf3-4308-a594-c558059b2c8b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:22.788242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1091/2179] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:22 2022] DELETE /identity/v3/users/a9f91915b7e54295b6dec28327db576c => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:22.790216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:22.800918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:22.801308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:22.804927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:22.805442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:22.805912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] PATH_INFO: `/v3/projects/e7453c1f59784041996c3372c732d899` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:22.812027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7453c1f59784041996c3372c732d899)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:22.822674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:22.968323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:23.031332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7453c1f59784041996c3372c732d899', 'request_id': 'req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.057096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7453c1f59784041996c3372c732d899', 'request_id': 'req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.070000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7453c1f59784041996c3372c732d899', 'request_id': 'req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.108407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] The token cache is being invalidate because project e7453c1f59784041996c3372c732d899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:23.108891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.110466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8ad525cd-68f6-4c20-8949-5e315f2bbd3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.113448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1089/2180] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:22 2022] DELETE /identity/v3/projects/e7453c1f59784041996c3372c732d899 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:23.118221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be4e43-f09c-4941-9ad9-09d6e87b8d1c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:23.118650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be4e43-f09c-4941-9ad9-09d6e87b8d1c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:23.119928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be4e43-f09c-4941-9ad9-09d6e87b8d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:23.128510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd67a2-3101-4416-b860-370b2387241e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:23.169942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f9be4e43-f09c-4941-9ad9-09d6e87b8d1c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:23.172347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f9be4e43-f09c-4941-9ad9-09d6e87b8d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:23.354611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:23.356473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:23.357470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:23.358323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:23.363154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:23.377956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1090/2181] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:23.381908 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:23.382601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:23.383182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:23.386744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:23.431349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:23.445400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:23.458428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1092/2182] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:22 2022] POST /identity/v3/users => generated 305 bytes in 668 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:23.461129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:23.465084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:23.500635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085c0e7-be39-4d66-a5c2-eb06bcc57914 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:23.514450 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:23.516376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:23.516891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:23.517464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] PATH_INFO: `/v3/projects/32fc8cf8feb741578e2669ae36b48ee8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:23.520242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32fc8cf8feb741578e2669ae36b48ee8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:23.524444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:23.533732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1091/2183] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:23.535306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:23.542722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:23.624043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:23.697169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32fc8cf8feb741578e2669ae36b48ee8', 'request_id': 'req-1d6533a1-233d-4768-9884-8150887f4e13'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.716515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32fc8cf8feb741578e2669ae36b48ee8', 'request_id': 'req-1d6533a1-233d-4768-9884-8150887f4e13'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.726277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32fc8cf8feb741578e2669ae36b48ee8', 'request_id': 'req-1d6533a1-233d-4768-9884-8150887f4e13'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.764717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] The token cache is being invalidate because project 32fc8cf8feb741578e2669ae36b48ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:23.765179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.766641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1845626-cc8f-4eb0-831f-a8e870dfc6a3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:23.767531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:23.770740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1093/2184] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:23 2022] DELETE /identity/v3/projects/32fc8cf8feb741578e2669ae36b48ee8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:23.772913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:23.777064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:23.976928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:23.978022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:23.993305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:23.995050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.000520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6533a1-233d-4768-9884-8150887f4e13 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.004146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:24.034043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.035700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.036342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.038565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.043671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.059799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.069007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73247ffd-ccb1-40a7-b977-f86496082ea7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.071165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.087155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1094/2185] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:24.093801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1092/2186] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:24.095531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.098716 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:24.100516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.101049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.104018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.156770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dec-f282-4e2b-b8d7-9456a5af95ff service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.172318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.174827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:24.176969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.177222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:24.177880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.178564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] PATH_INFO: `/v3/users/ce9e1746a4fd4b0680edfbbbadeee21f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.182146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9e1746a4fd4b0680edfbbbadeee21f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.185137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.194596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:24.196114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1095/2187] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:24.199897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.226556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079fbb-f440-4839-8431-bc6c793ecaad None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.236724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.238761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.238952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.239480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.241277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.244928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.280129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1096/2188] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:24.282510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.305927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdde570-f12a-4f06-b3ba-534ab6f0c693 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.319814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.323127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.323127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.323501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.326516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:24.327255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.332849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.348056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9e1746a4fd4b0680edfbbbadeee21f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:24.369053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9e1746a4fd4b0680edfbbbadeee21f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:24.377676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1097/2189] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:24.377676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.400472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4207-7f61-4e5f-b363-a56f4c389da9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.401703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9e1746a4fd4b0680edfbbbadeee21f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:24.410068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.412350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.413220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.417441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] PATH_INFO: `/v3/projects/ea6c61ae6f344795a0002ab8c5c94094/users/e51df6c85c7d447e9cc1a1339b6f4b77/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.464511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9e1746a4fd4b0680edfbbbadeee21f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:24.465983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9e1746a4fd4b0680edfbbbadeee21f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:24.477104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6c61ae6f344795a0002ab8c5c94094, user_id=e51df6c85c7d447e9cc1a1339b6f4b77, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.484356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9091-941a-419a-9261-2cdbd380e940 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.498239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1093/2190] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:24 2022] DELETE /identity/v3/users/ce9e1746a4fd4b0680edfbbbadeee21f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:24.498239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.505209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:24.570386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1098/2191] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:24 2022] PUT /identity/v3/projects/ea6c61ae6f344795a0002ab8c5c94094/users/e51df6c85c7d447e9cc1a1339b6f4b77/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:24.580729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.582449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.582926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.701140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088cf7e-591f-466f-8cb4-6d808d28c054 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:24.717877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] MFA Rules not processed for user `e51df6c85c7d447e9cc1a1339b6f4b77`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:24.722559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:24.892859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:24.972948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:24.974461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:24.974982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:24.975464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:24.976845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:24.981841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1099/2192] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:24.983603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:24.985676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:24.987219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:25.025457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa3aea-fed6-464c-a39f-ce5dad97c08f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.041122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.042379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.042948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.043431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.045968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.059082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.065511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1094/2193] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:24 2022] POST /identity/v3/projects => generated 377 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:25.070553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.128625 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:25.149945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1100/2194] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:24 2022] POST /identity/v3/projects => generated 409 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:25.158902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.327256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bbb82-7d62-40a9-a699-a2a8137256b3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.401058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15a21-cad0-4306-a5e5-fbb2429b1ccf admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.432837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.443262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.443906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.444480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.447668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.453822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.500339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1101/2195] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:25 2022] POST /identity/v3/users => generated 311 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:25.508388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.516691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.518081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.518658 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.519258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.521168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.528246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04c69-8551-4141-a19b-f1194babb929 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.541497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.541497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.541497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.542123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.544009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.548356 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.550898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.574473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1102/2196] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:25.581114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.584473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:25.644183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf92b7-83af-4e85-a41c-98707762199f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.659060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.659060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.659060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.659060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] PATH_INFO: `/v3/projects/c277be7923c2427c860030ab31a64b96` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.659703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c277be7923c2427c860030ab31a64b96)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.663723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.678032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1095/2197] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:25 2022] POST /identity/v3/users => generated 289 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:25.691761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.725714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06681120-7fe1-49e8-8543-2d987c5561f3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:25.741224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:25.745049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:25.745049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:25.745049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] PATH_INFO: `/v3/projects/4955cea805e646ba9b5b211fed96ff81/users/6b9db5837ad948ecad8eec7190fe4b49/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:25.797122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c277be7923c2427c860030ab31a64b96', 'request_id': 'req-3a5dc785-ba08-4fd9-b012-c54582d2b30b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:25.803936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4955cea805e646ba9b5b211fed96ff81, user_id=6b9db5837ad948ecad8eec7190fe4b49, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:25.809321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:25.819413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c277be7923c2427c860030ab31a64b96', 'request_id': 'req-3a5dc785-ba08-4fd9-b012-c54582d2b30b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:25.837487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1096/2198] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:25 2022] PUT /identity/v3/projects/4955cea805e646ba9b5b211fed96ff81/users/6b9db5837ad948ecad8eec7190fe4b49/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:25.842425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.854539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c277be7923c2427c860030ab31a64b96', 'request_id': 'req-3a5dc785-ba08-4fd9-b012-c54582d2b30b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:25.891176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] The token cache is being invalidate because project c277be7923c2427c860030ab31a64b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:25.891970 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:25.893724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:25.910504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1103/2199] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:25 2022] DELETE /identity/v3/projects/c277be7923c2427c860030ab31a64b96 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:25.910504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:25.911344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:25.994705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:26.038691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71925d7-22a1-4bd6-acaf-a15327568895 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.047732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.049710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.051200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.052164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.054292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.057577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.081690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1097/2200] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:26.083595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.087768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.128424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16c5c-3acb-4068-96a9-6a1b8f14d261 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.137246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.138151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dc785-ba08-4fd9-b012-c54582d2b30b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.138536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.139211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.139520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.140929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.144388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.162367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.169200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.169820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1098/2201] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:26.169968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.169968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] PATH_INFO: `/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.170240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.176842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.192828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6d6b7a66fc472096c3d3a457b7a26b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.200292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dac91-faac-4806-b48b-39d490ff351b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.207556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.209379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.210901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.211365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.211891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] PATH_INFO: `/v3/users/7f564d9a814d493a88461f433c4dc317` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.214742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f564d9a814d493a88461f433c4dc317)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.219369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.232141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:26.362805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f564d9a814d493a88461f433c4dc317'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.388093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f564d9a814d493a88461f433c4dc317'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.405073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6d6b7a66fc472096c3d3a457b7a26b', 'request_id': 'req-992abc42-d6da-4291-9696-018ca1fcf22f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.422991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6d6b7a66fc472096c3d3a457b7a26b', 'request_id': 'req-992abc42-d6da-4291-9696-018ca1fcf22f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.429049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f564d9a814d493a88461f433c4dc317'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.439942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6d6b7a66fc472096c3d3a457b7a26b', 'request_id': 'req-992abc42-d6da-4291-9696-018ca1fcf22f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.461978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f564d9a814d493a88461f433c4dc317'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.463011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f564d9a814d493a88461f433c4dc317'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.468346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] The token cache is being invalidate because project 7d6d6b7a66fc472096c3d3a457b7a26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:26.468704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.469641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:26.472226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1104/2202] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:25 2022] DELETE /identity/v3/projects/7d6d6b7a66fc472096c3d3a457b7a26b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:26.473883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.476098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.482778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1099/2203] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:26 2022] DELETE /identity/v3/users/7f564d9a814d493a88461f433c4dc317 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:26.483800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.485906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.648841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992abc42-d6da-4291-9696-018ca1fcf22f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.670286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.672054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.672325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.672850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.674534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.679504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.683294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0a1aa-636e-408e-ad5b-bd2dc5f24ea4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.692854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:26.694856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.695397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.695894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] PATH_INFO: `/v3/projects/4955cea805e646ba9b5b211fed96ff81/users/6b9db5837ad948ecad8eec7190fe4b49/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.700870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4955cea805e646ba9b5b211fed96ff81, user_id=6b9db5837ad948ecad8eec7190fe4b49, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:26.706729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99b8e9-fded-4e24-a9f0-6ae06333b23b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:26.725313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1105/2204] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:26.727307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.727509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1100/2205] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:26 2022] PUT /identity/v3/projects/4955cea805e646ba9b5b211fed96ff81/users/6b9db5837ad948ecad8eec7190fe4b49/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:26.730539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:26.731171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:26.731721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:26.734178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.779458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:26.781602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:26.784530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fcc3-fa12-417e-b4b2-5f6e0aa97650 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:26.963573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:26.970298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1101/2206] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:26.971769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:26.992364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5304e89-0a79-4d9c-952e-3cde99740287 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:27.003424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:27.005135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.005833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.006259 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] PATH_INFO: `/v3/projects/a54ebe7b1a3a4cd58e2dd2e9043c2dff/users/ad574c124f8e4e6ca49a9539fca8715c/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.016666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:27.020047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.020744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.021611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] PATH_INFO: `/v3/users/c341cb341b6048458d16665a318da37b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.027082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c341cb341b6048458d16665a318da37b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:27.032615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54ebe7b1a3a4cd58e2dd2e9043c2dff, user_id=ad574c124f8e4e6ca49a9539fca8715c, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:27.033729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:27.035932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b35246-ca2f-4824-a30b-db31a9c901c0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:27.040374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:27.072710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1102/2207] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:26 2022] PUT /identity/v3/projects/a54ebe7b1a3a4cd58e2dd2e9043c2dff/users/ad574c124f8e4e6ca49a9539fca8715c/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:27.074270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfef14c-0017-44d5-9154-49c71086e834 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.074564 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfef14c-0017-44d5-9154-49c71086e834 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.074838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfef14c-0017-44d5-9154-49c71086e834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.150300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-acfef14c-0017-44d5-9154-49c71086e834 None None] MFA Rules not processed for user `6b9db5837ad948ecad8eec7190fe4b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:27.152204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-acfef14c-0017-44d5-9154-49c71086e834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.171233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c341cb341b6048458d16665a318da37b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.198889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c341cb341b6048458d16665a318da37b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.199747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c341cb341b6048458d16665a318da37b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.218610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c341cb341b6048458d16665a318da37b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.219421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-89f9a261-e5fd-4b3a-a0a4-032c124030f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c341cb341b6048458d16665a318da37b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.267231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1106/2208] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:26 2022] DELETE /identity/v3/users/c341cb341b6048458d16665a318da37b => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:27.270442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bfb78-6473-4bf3-a2b5-aa80dbd29103 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.272319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bfb78-6473-4bf3-a2b5-aa80dbd29103 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.273497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bfb78-6473-4bf3-a2b5-aa80dbd29103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.293011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1103/2209] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:27.293011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.293011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.293011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.314165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7a0bfb78-6473-4bf3-a2b5-aa80dbd29103 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:27.317236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7a0bfb78-6473-4bf3-a2b5-aa80dbd29103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.336688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] MFA Rules not processed for user `ad574c124f8e4e6ca49a9539fca8715c`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:27.338969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.502654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1107/2210] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:27.504481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.504874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.505457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.565430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1104/2211] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:27.572857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:27.575568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.576854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:27.578101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.587684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1108/2212] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:27.587980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:27.591843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.639821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d919-30f7-4544-98f9-6e3582924e8c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:27.650681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:27.651860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.652270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.652717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] PATH_INFO: `/v3/projects/41affccca2c84fc8ba2d494f6cafe1fa` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.654964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41affccca2c84fc8ba2d494f6cafe1fa)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:27.657958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:27.701425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b4e7c-078d-4250-b0b3-29ca24e4ed1b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:27.707083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:27.707869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.708217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.708691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.779624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:27.785304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41affccca2c84fc8ba2d494f6cafe1fa', 'request_id': 'req-80584531-c069-4579-a0be-456494c68c6c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.800224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41affccca2c84fc8ba2d494f6cafe1fa', 'request_id': 'req-80584531-c069-4579-a0be-456494c68c6c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.805839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:27.830827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41affccca2c84fc8ba2d494f6cafe1fa', 'request_id': 'req-80584531-c069-4579-a0be-456494c68c6c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.837107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:27.839586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:27.865872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1105/2213] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:27.866202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] The token cache is being invalidate because project 41affccca2c84fc8ba2d494f6cafe1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:27.866696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:27.867359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.869016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:27.877072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1109/2214] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:27 2022] DELETE /identity/v3/projects/41affccca2c84fc8ba2d494f6cafe1fa => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:27.878796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:27.881514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:27.881667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf403-cfa1-49b0-92e8-221d843995e4 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:27.991990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:27.993433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:27.993970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:27.994140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:27.995620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:27.996175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:27.999343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.020705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1106/2215] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:28.022670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.032212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.110435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80584531-c069-4579-a0be-456494c68c6c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.133308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.134231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.134773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.135716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.138127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.142538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.176887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0d6c-a187-43a7-988a-cb5537a20a2f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.185587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.186957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.187450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.187783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.192432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.208269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.210741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.231501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1110/2216] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:27 2022] POST /identity/v3/projects => generated 381 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:28.233302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.236720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.249838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1107/2217] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:28.249838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.261729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.292974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387521d-b800-4801-b55b-c1017c8440b3 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.385222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbadf9-2fd9-49ae-96c7-00df948faffb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.388481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:28.394716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.396184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.396656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.397208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] PATH_INFO: `/v3/users/714dae18b6c441b4b31dabecc7058859` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.399445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=714dae18b6c441b4b31dabecc7058859)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.403039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.410959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.410959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.410959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.410959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.412511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.416435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.481364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1108/2218] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:28 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:28.489288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.511574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714dae18b6c441b4b31dabecc7058859'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.525534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714dae18b6c441b4b31dabecc7058859'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.527045 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714dae18b6c441b4b31dabecc7058859'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.543449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714dae18b6c441b4b31dabecc7058859'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.544091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714dae18b6c441b4b31dabecc7058859'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.558337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:28.582775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1111/2219] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:28 2022] DELETE /identity/v3/users/714dae18b6c441b4b31dabecc7058859 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:28.584726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.586586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.620343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d135c-0fa5-4fe1-ac59-cd19a95a98d2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.726978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bde6d-2379-47ca-b92b-eeade6d4ed2c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.736545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.745070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.746218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.746470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.746674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.746928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.747123 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.747375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.748303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.752302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.756534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.788008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1109/2220] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:28 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:28.789305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.792078 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.812513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c1f8-448d-40f2-add6-0e58eadf1ce9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.821013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.821876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.822384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.822957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.825290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.828503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.865864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1110/2221] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:28 2022] POST /identity/v3/users => generated 292 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:28.865864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.868220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.883580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662e71b-926f-42e1-a286-07e3c40f62a0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.883814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.885445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.890501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.892093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.892486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.892838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] PATH_INFO: `/v3/projects/62fd23a9b75c45398b6c73c46f80d96d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.894240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=62fd23a9b75c45398b6c73c46f80d96d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.896436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.902118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1112/2222] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:28 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:28.903639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.906500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:28.935641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832e81-d5fc-411b-b135-c0ae2f2ce001 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:28.944327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:28.945003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:28.945480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:28.945921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:28.947168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:28.950479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:28.970475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1113/2223] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:28.971744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:28.984887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62fd23a9b75c45398b6c73c46f80d96d', 'request_id': 'req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:28.988959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f0f4-0962-4d7e-9422-d78279690b23 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.001906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62fd23a9b75c45398b6c73c46f80d96d', 'request_id': 'req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.004909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:INSERT=8,keystone:DELETE=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:29.017028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62fd23a9b75c45398b6c73c46f80d96d', 'request_id': 'req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.054905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] The token cache is being invalidate because project 62fd23a9b75c45398b6c73c46f80d96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:29.055276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.056010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.058121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1111/2224] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:28 2022] DELETE /identity/v3/projects/62fd23a9b75c45398b6c73c46f80d96d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:29.059774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.063350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:29.150376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.151763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.152476 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.153369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.159216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.161580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.199319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1114/2225] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:29.200544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.209796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:29.240223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba641d5c-eada-463e-ba0f-2bb8811c45cb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.252220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.255204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.255496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.255998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] PATH_INFO: `/v3/users/79440e3d116f43b29f0a2da5a735ddfe` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.259781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=79440e3d116f43b29f0a2da5a735ddfe)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.263974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.268129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e9f8-c46d-4959-a985-cfb32ddc6e1e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.285671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.286171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.286831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.287400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] PATH_INFO: `/v3/projects/88d312b902f24156bb3207e94731de6b/users/b045fae564da4c7a8047939fa848fc4a/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.339475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:29.341226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d312b902f24156bb3207e94731de6b, user_id=b045fae564da4c7a8047939fa848fc4a, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.346523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.401775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1112/2226] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:29 2022] PUT /identity/v3/projects/88d312b902f24156bb3207e94731de6b/users/b045fae564da4c7a8047939fa848fc4a/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:29.401775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.415581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:29.426143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79440e3d116f43b29f0a2da5a735ddfe'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.453695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79440e3d116f43b29f0a2da5a735ddfe'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.462290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79440e3d116f43b29f0a2da5a735ddfe'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.489352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79440e3d116f43b29f0a2da5a735ddfe'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.491460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79440e3d116f43b29f0a2da5a735ddfe'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.573217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1115/2227] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:29 2022] DELETE /identity/v3/users/79440e3d116f43b29f0a2da5a735ddfe => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:29.577111 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.577752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:29.655181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d2229-8d8b-4eda-a756-3acf12931fc4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.787129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc1498-65ad-4021-b0f7-64e4cabdcdaa admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.797789 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.800254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.801487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.802317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.804606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.808131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.808715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.808991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.809198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] PATH_INFO: `/v3/projects/98599a877a51432f91cd9ea024a52407` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.810745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98599a877a51432f91cd9ea024a52407)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.811167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.813593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.831265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1116/2228] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:29.834394 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.838730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:29.839619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:29.861387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78f59b-85d1-4db1-9c9c-d2f5c094b77d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.869646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:29.870715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:29.871546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:29.871932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:29.873433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:29.875558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:29.889795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1117/2229] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:29.891467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.892064 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98599a877a51432f91cd9ea024a52407', 'request_id': 'req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.907321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98599a877a51432f91cd9ea024a52407', 'request_id': 'req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.911224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5644acd-8d4b-47cb-a1f4-6578ee0ea2ce admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:29.921546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98599a877a51432f91cd9ea024a52407', 'request_id': 'req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.944941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] The token cache is being invalidate because project 98599a877a51432f91cd9ea024a52407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:29.945538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.946492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:29.949150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1113/2230] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:29 2022] DELETE /identity/v3/projects/98599a877a51432f91cd9ea024a52407 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:29.955797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:29.957554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.038470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:30.040129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.040732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.041081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] PATH_INFO: `/v3/projects/966cf8bbbc4540a596f615498331465a/users/55d7bbda47b64493a840fef8c6c40431/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.076029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966cf8bbbc4540a596f615498331465a, user_id=55d7bbda47b64493a840fef8c6c40431, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:30.081031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30ba85-2d77-4925-9e67-86d484d7ddf8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:30.133371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1118/2231] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:29 2022] PUT /identity/v3/projects/966cf8bbbc4540a596f615498331465a/users/55d7bbda47b64493a840fef8c6c40431/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:30.133371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ddc7-bbda-4ee8-8ba5-f78a3f4799d4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.133371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ddc7-bbda-4ee8-8ba5-f78a3f4799d4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.133371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ddc7-bbda-4ee8-8ba5-f78a3f4799d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.166443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0f5c-0a87-4ede-93a1-2b5f21f5c4f1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:30.169249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7088ddc7-bbda-4ee8-8ba5-f78a3f4799d4 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:30.170741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7088ddc7-bbda-4ee8-8ba5-f78a3f4799d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.178872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:30.273370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:30.274493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.274875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.275186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] PATH_INFO: `/v3/projects/88d312b902f24156bb3207e94731de6b/users/b045fae564da4c7a8047939fa848fc4a/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.277652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d312b902f24156bb3207e94731de6b, user_id=b045fae564da4c7a8047939fa848fc4a, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:30.281322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151fd39-05ba-49d0-a5b1-2903f207ba8f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:30.297261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1114/2232] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:29 2022] PUT /identity/v3/projects/88d312b902f24156bb3207e94731de6b/users/b045fae564da4c7a8047939fa848fc4a/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:30.298714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebae9-5b1f-4adc-8997-622dccc0691b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.299009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebae9-5b1f-4adc-8997-622dccc0691b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.299289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebae9-5b1f-4adc-8997-622dccc0691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.318932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1119/2233] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:30.321319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.321726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.321999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.326107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-91eebae9-5b1f-4adc-8997-622dccc0691b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:30.327694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-91eebae9-5b1f-4adc-8997-622dccc0691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.394356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:30.403570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] MFA Rules not processed for user `b045fae564da4c7a8047939fa848fc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:30.405746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.426866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1115/2234] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:30.428837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.429304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.429752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.477560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:30.480197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.495284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1116/2235] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:30.503131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:30.506512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.528574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac6170-2aea-4453-9120-be2434f12686 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:30.534658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1120/2236] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:30.537886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:30.540172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.540725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.541261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] PATH_INFO: `/v3/projects/7406d087e15b44c79f95fa43baa9689b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.543901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7406d087e15b44c79f95fa43baa9689b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:30.547828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:30.548162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:30.550386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.637449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7406d087e15b44c79f95fa43baa9689b', 'request_id': 'req-d860dff0-5df3-4428-9cd6-532af6576bb9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:30.653031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7406d087e15b44c79f95fa43baa9689b', 'request_id': 'req-d860dff0-5df3-4428-9cd6-532af6576bb9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:30.667497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7406d087e15b44c79f95fa43baa9689b', 'request_id': 'req-d860dff0-5df3-4428-9cd6-532af6576bb9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:30.668874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416a54d-5f58-4c89-aba7-d79622fd6f2d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:30.693550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] The token cache is being invalidate because project 7406d087e15b44c79f95fa43baa9689b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:30.694227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:30.695976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:30.699501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1117/2237] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:30 2022] DELETE /identity/v3/projects/7406d087e15b44c79f95fa43baa9689b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:30.702912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:30.712167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.749431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:30.750506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.751163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.751429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.754726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.768745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:30.866238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dff0-5df3-4428-9cd6-532af6576bb9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:30.874643 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:30.875886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.876491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.877068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.878669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:30.879307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:30.883025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:30.915804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1118/2238] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:30.915804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:30.918749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:30.929670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:30.931770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466ce87-c707-40c1-8ebd-845cf67a0243 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:30.943740 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1121/2239] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:30.945121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:30.945449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:30.945734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:30.975056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:30.977103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:31.001826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1122/2240] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:31.001826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.019040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b419d-e08a-49d1-a564-b430cb50b44b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.025209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.026105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.026385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.026670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.027496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.030304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75ef30-c9dd-47b9-a7f1-dbdc8966ffdb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.072718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeaa9e-6847-4cf5-9659-e68675441b3b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.079107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.079984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.080375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.080758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.087385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1123/2241] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:30 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:31.087649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:31.087840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.087840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.087840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.105213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:31.126327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:31.127226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:31.196261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.198441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.224724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1119/2242] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:31.229068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.266638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b43bd2-5c4e-480a-a2c7-877e85d0e2f2 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.372850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1124/2243] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:31.378962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.392342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867bdf3-f862-489d-a3f3-ea0b95963f84 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.400072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.401762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.402489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.403104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.404103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:31.404580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.406542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.432883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1125/2244] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:31.434997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.463823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1990-6b8d-47d2-b6dd-b5ed98f4c007 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.477584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.477584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.477584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.477584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.478325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.478325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.484776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.490790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.491435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.492105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.493457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.499503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.557555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1126/2245] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:31 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:31.561408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.573689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1120/2246] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:31 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:31.575774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.592630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057b6e-4894-4856-858a-4507945ed20e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.659966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:31.691395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.694161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.694701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.695081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.697879 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.702848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.719246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1121/2247] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:31.725061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.733479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd73eda-8ccd-4002-af2f-cbe7027913d1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.739548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78d2c3-b9be-4ccc-9726-4b4082bb811e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.745456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.746266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.746555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.746824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.747749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.749799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.758321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.759538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.760055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.760998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.762034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.764350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1122/2248] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:31.770019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.771453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.782647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02115-23ac-4adf-971c-7bf5f3b9ca57 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.790066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:31.791096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.791384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.791669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] PATH_INFO: `/v3/projects/b2849fb2c31840599e5cae234e57bf88/users/d47779fd181046a199fd8400d70485d1/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.821147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2849fb2c31840599e5cae234e57bf88, user_id=d47779fd181046a199fd8400d70485d1, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:31.824428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a486dde-b8a1-4a64-88d0-aacf9034f8a8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:31.839142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:31.851920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1127/2250] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:31 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:31.854138 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:31.857620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1123/2250] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:31 2022] PUT /identity/v3/projects/b2849fb2c31840599e5cae234e57bf88/users/d47779fd181046a199fd8400d70485d1/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:31.859960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:31.860257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:31.860576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:31.904495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c541-6662-4001-b36b-1dab7f0ff922 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:31.916217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] MFA Rules not processed for user `d47779fd181046a199fd8400d70485d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:31.917944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:32.027010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1124/2251] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:32.073074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.122792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.136432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.136432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.136432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.136432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.136432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.174870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1128/2252] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:32.177599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.254128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a380f9-b7bf-47db-af58-c9ea9a3ec8c7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.265511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.268137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.268584 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.268972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.295907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.300448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.350163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1125/2253] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:26:32.350163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.350781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d8280-1d1f-4023-897a-3b1e99906411 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.365867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b045fae564da4c7a8047939fa848fc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d312b902f24156bb3207e94731de6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.365867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.365867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.366662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.366662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:32.366662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.366662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.389002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5543295-41a6-4be7-b04b-64187e57e193 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.396444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1129/2254] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:32.398134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.403730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.404425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.404822 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.405101 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.405932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.409585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.418284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde228-d5d1-479d-b503-ddc6667e03ff tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.427548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b045fae564da4c7a8047939fa848fc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d312b902f24156bb3207e94731de6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.428718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.429202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.429730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.431396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.444116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1126/2255] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:32.445848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.446051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.483912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eead36-0906-49e8-95e5-0728baae6387 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.502024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:32.505647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.506103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.508652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] PATH_INFO: `/v3/projects/4318b5b98e084c678ce43a44e8fefccb/users/871dd9685c5e45a88a4c6e8be052f686/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.528291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1130/2256] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:32 2022] POST /identity/v3/projects => generated 355 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:32.536352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.550701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4318b5b98e084c678ce43a44e8fefccb, user_id=871dd9685c5e45a88a4c6e8be052f686, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:32.555937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee90e1-f4f6-4a73-839f-455464bd2c5e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:32.588927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1127/2257] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:32 2022] PUT /identity/v3/projects/4318b5b98e084c678ce43a44e8fefccb/users/871dd9685c5e45a88a4c6e8be052f686/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:32.596439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:32.597122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:32.597589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:32.653073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:32.662524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:32.693506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:32.800314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f9fe8-fca7-42f9-96c9-83b958c42f80 tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:32.819219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:32.868905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1128/2258] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:32.887542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:32.897722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:33.026864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b045fae564da4c7a8047939fa848fc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d312b902f24156bb3207e94731de6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:33.029046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:33.031508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:33.031971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:33.032962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:33.035324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:33.037101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:33.076342 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1131/2259] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:33.081970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9d2b6-641e-42a6-9312-89bac1e52b63 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:33.105853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:33.107014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:33.107468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:33.108341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:33.141976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:33.144435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:33.167840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:33.169720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1129/2260] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:26:33.183570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e79-7bd0-4320-9698-00c98380360a tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:33.190165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b045fae564da4c7a8047939fa848fc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88d312b902f24156bb3207e94731de6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:33.191031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:33.191316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:33.191601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] PATH_INFO: `/v3/projects/3d391b2c267d4f9e95a3cee1d6232022` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:33.193703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3d391b2c267d4f9e95a3cee1d6232022)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:33.195909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:33.275289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d391b2c267d4f9e95a3cee1d6232022', 'request_id': 'req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.284484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d391b2c267d4f9e95a3cee1d6232022', 'request_id': 'req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.294416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d391b2c267d4f9e95a3cee1d6232022', 'request_id': 'req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.319228 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] The token cache is being invalidate because project 3d391b2c267d4f9e95a3cee1d6232022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:33.320890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.322667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8e1eb0ea-52ce-4ad1-9657-13aa509f834e tempest-VolumeQuotaClassesTest-1423499522 tempest-VolumeQuotaClassesTest-1423499522-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.325822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1132/2261] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:33 2022] DELETE /identity/v3/projects/3d391b2c267d4f9e95a3cee1d6232022 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:33.705230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:33.707700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:33.730150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c56aa-3233-450c-bb40-f83687869d76 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:33.826709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:33.827653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:33.829034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:33.829034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] PATH_INFO: `/v3/users/ad574c124f8e4e6ca49a9539fca8715c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:33.830210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad574c124f8e4e6ca49a9539fca8715c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:33.833399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:33.938711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad574c124f8e4e6ca49a9539fca8715c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.959136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad574c124f8e4e6ca49a9539fca8715c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:33.979135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad574c124f8e4e6ca49a9539fca8715c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:34.016172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad574c124f8e4e6ca49a9539fca8715c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:34.017756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad574c124f8e4e6ca49a9539fca8715c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:34.043330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1130/2262] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:33 2022] DELETE /identity/v3/users/ad574c124f8e4e6ca49a9539fca8715c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:34.052039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:34.052883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:34.053498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:34.105019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:34.107442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:34.246681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1133/2263] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:34.269657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:34.278130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:34.330707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:34.442990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb8d0b-ffe2-48f6-85bb-6be4e2fec660 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:34.452226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:34.455073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:34.455793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:34.456824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:34.476947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:34.480029 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd53cbc-6330-44a8-bae8-2dddabab7310 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:34.537091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1131/2264] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:35.042299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:35.043971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:35.064849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:35.074290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30520b90-f8c3-4e92-bbf3-2a768bf94221 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:35.084783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:35.086054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:35.087637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:35.088086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] PATH_INFO: `/v3/users/d47779fd181046a199fd8400d70485d1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:35.091811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47779fd181046a199fd8400d70485d1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:35.095941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:35.198722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47779fd181046a199fd8400d70485d1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:35.209903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47779fd181046a199fd8400d70485d1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:35.211966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47779fd181046a199fd8400d70485d1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:35.223054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47779fd181046a199fd8400d70485d1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:35.224186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47779fd181046a199fd8400d70485d1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:35.283443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1134/2265] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:35 2022] DELETE /identity/v3/users/d47779fd181046a199fd8400d70485d1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:35.286534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:35.318169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:35.318663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:35.434622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:35.437011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:35.664655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1132/2266] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:35.691837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:35.694142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:35.915848 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94554059-efc9-4770-a692-ff5336da1667 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:35.924110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:35.925793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:35.926693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:35.926981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:35.927556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:35.930135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:35.950546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:35.952577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:35.969073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1135/2267] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:35.982665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74741-f5b9-4287-a50b-bc9a82743fc7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:36.007826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:36.008792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:36.011595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:36.012014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] PATH_INFO: `/v3/projects/a54ebe7b1a3a4cd58e2dd2e9043c2dff` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:36.016102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54ebe7b1a3a4cd58e2dd2e9043c2dff)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:36.018357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:36.136048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54ebe7b1a3a4cd58e2dd2e9043c2dff', 'request_id': 'req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.165938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54ebe7b1a3a4cd58e2dd2e9043c2dff', 'request_id': 'req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.180506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54ebe7b1a3a4cd58e2dd2e9043c2dff', 'request_id': 'req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.201318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] The token cache is being invalidate because project a54ebe7b1a3a4cd58e2dd2e9043c2dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:36.201318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.201995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bac3cd05-1c8c-42ae-9d2d-f0756bb2a04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.208913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:36.219220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1133/2268] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:35 2022] DELETE /identity/v3/projects/a54ebe7b1a3a4cd58e2dd2e9043c2dff => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:36.227062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2049-67e8-4bb1-b9c0-b0be028e1d07 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:36.385308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:36.386695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:36.387164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:36.387586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] PATH_INFO: `/v3/users/b045fae564da4c7a8047939fa848fc4a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:36.389296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b045fae564da4c7a8047939fa848fc4a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:36.397411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:36.441795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:36.442642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:36.442642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:36.465579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:36.523065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] MFA Rules not processed for user `871dd9685c5e45a88a4c6e8be052f686`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:36.525375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:36.530809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b045fae564da4c7a8047939fa848fc4a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.546559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b045fae564da4c7a8047939fa848fc4a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.546559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b045fae564da4c7a8047939fa848fc4a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.566622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b045fae564da4c7a8047939fa848fc4a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.566622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b045fae564da4c7a8047939fa848fc4a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:36.610544 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1136/2269] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:36 2022] DELETE /identity/v3/users/b045fae564da4c7a8047939fa848fc4a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:36.642334 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:36.743554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1134/2270] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:36.748932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:36.751675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:37.148302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3376d-a155-40f4-8ff3-fc4033752c08 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:37.160329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:37.161148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:37.161502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:37.161803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:37.162540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:37.167646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:37.245561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1137/2271] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:36 2022] POST /identity/v3/projects => generated 373 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:37.257077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:37.305073 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e321991-92c8-4dd6-b4dd-a2ab8036dc83 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:37.566264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:37.570320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:37.571010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:37.572167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:37.576756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:37.586329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:37.682888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1135/2272] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:37 2022] POST /identity/v3/users => generated 293 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:37.690668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:37.742390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ccc-0d53-4657-bf3b-104ab52c92e3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:37.771552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:37.774510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:37.777656 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:37.779609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:37.782400 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:37.789440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:37.850863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:37.853876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1138/2273] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:37.867061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392277-7ac0-47e7-b637-6374b3b08baf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:37.882387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:37.883612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:37.884011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:37.884441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] PATH_INFO: `/v3/projects/2c6c1bc01f754751a8330d6cf311a30f/users/15de7bbd869f41baa4e21690c79150dc/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:37.891454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:37.895572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:37.923667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6c1bc01f754751a8330d6cf311a30f, user_id=15de7bbd869f41baa4e21690c79150dc, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:37.926718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:37.967233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305d67-4397-48c5-9350-36e2a77f9880 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:37.967926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1136/2274] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:37 2022] PUT /identity/v3/projects/2c6c1bc01f754751a8330d6cf311a30f/users/15de7bbd869f41baa4e21690c79150dc/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:37.969602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:38.234355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:38.235692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:38.236224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:38.236647 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] PATH_INFO: `/v3/projects/b2849fb2c31840599e5cae234e57bf88` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:38.241322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2849fb2c31840599e5cae234e57bf88)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:38.245320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:38.265852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428239c-89cb-44ca-928f-642b99da872b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:38.291563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:38.295930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:38.298109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:38.298920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:38.301499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:38.303577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:38.306065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:38.345831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1137/2275] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:38.357881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:38.386067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9f653-261f-4929-a7b5-e7ffc477e995 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:38.439481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2849fb2c31840599e5cae234e57bf88', 'request_id': 'req-1dd384be-075d-484d-b517-2d101a847030'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:38.474618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2849fb2c31840599e5cae234e57bf88', 'request_id': 'req-1dd384be-075d-484d-b517-2d101a847030'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:38.506664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2849fb2c31840599e5cae234e57bf88', 'request_id': 'req-1dd384be-075d-484d-b517-2d101a847030'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:38.510952 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:38.581486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] The token cache is being invalidate because project b2849fb2c31840599e5cae234e57bf88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:38.582591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:38.586063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1dd384be-075d-484d-b517-2d101a847030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:38.589980 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1139/2276] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:37 2022] DELETE /identity/v3/projects/b2849fb2c31840599e5cae234e57bf88 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:38.598392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:38.599008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:38.599712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:38.625676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:38.627179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:38.631506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:38.632142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] PATH_INFO: `/v3/projects/2c6c1bc01f754751a8330d6cf311a30f/users/15de7bbd869f41baa4e21690c79150dc/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:38.643433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6c1bc01f754751a8330d6cf311a30f, user_id=15de7bbd869f41baa4e21690c79150dc, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:38.658757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc6349-0048-47f2-912d-a9a7ad8d447a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:38.659173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:38.662565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:38.680755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1140/2277] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:38.682541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:38.686203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:38.708594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1138/2278] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:38 2022] PUT /identity/v3/projects/2c6c1bc01f754751a8330d6cf311a30f/users/15de7bbd869f41baa4e21690c79150dc/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:38.714479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:38.715047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:38.717614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:38.720810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b077f30-3883-4cde-92f5-7692fe9be8c6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:38.788639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:38.799770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.010671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1139/2279] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:39.012900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:39.016314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.019193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:39.021793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.022376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.022810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] PATH_INFO: `/v3/projects/88d312b902f24156bb3207e94731de6b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.025326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d312b902f24156bb3207e94731de6b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:39.029860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:39.045819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104f36-b69a-4572-a793-8594fb43e0ae None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:39.055616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:39.060011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.061054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.061993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] PATH_INFO: `/v3/users/e51df6c85c7d447e9cc1a1339b6f4b77` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.065647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51df6c85c7d447e9cc1a1339b6f4b77)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:39.069726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:39.179809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d312b902f24156bb3207e94731de6b', 'request_id': 'req-36c502f7-1fd7-4c30-8598-fe316393049d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.200448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d312b902f24156bb3207e94731de6b', 'request_id': 'req-36c502f7-1fd7-4c30-8598-fe316393049d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.204986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51df6c85c7d447e9cc1a1339b6f4b77'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.224033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51df6c85c7d447e9cc1a1339b6f4b77'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.255717 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51df6c85c7d447e9cc1a1339b6f4b77'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.267206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d312b902f24156bb3207e94731de6b', 'request_id': 'req-36c502f7-1fd7-4c30-8598-fe316393049d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.306954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51df6c85c7d447e9cc1a1339b6f4b77'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.308519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f8f1ffe5-bd98-45f0-b4b6-13646d79396d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51df6c85c7d447e9cc1a1339b6f4b77'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.309576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] The token cache is being invalidate because project 88d312b902f24156bb3207e94731de6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:39.310359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.314069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:39.323572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1141/2280] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:38 2022] DELETE /identity/v3/projects/88d312b902f24156bb3207e94731de6b => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:39.327142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:39.331634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.339997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1140/2281] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:39 2022] DELETE /identity/v3/users/e51df6c85c7d447e9cc1a1339b6f4b77 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:39.344832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4b8a-6975-49c5-a522-e19e601a4f6a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.344832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4b8a-6975-49c5-a522-e19e601a4f6a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.344832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4b8a-6975-49c5-a522-e19e601a4f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.485864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:39.494462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7e5c4b8a-6975-49c5-a522-e19e601a4f6a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:39.498825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7e5c4b8a-6975-49c5-a522-e19e601a4f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.615084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c502f7-1fd7-4c30-8598-fe316393049d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:39.645039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:39.705902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1141/2282] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:39.709215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.709215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.709823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.744751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:39.745601 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.745891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.746165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.747008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:39.747319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:39.749410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75614349-45af-4520-b97f-dbd9c0127dc9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:39.753600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:39.758445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.774850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1142/2283] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:39.777689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.778359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1142/2284] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:39.778858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.779633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.781674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:39.816948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449fd01-5ff8-477f-b994-732997fbb12d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:39.833226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:39.833993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:39.834685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:39.834966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:39.835244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:39.836124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.836413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:39.839322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:39.860819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1143/2285] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:39.860819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:39.862553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:39.893383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1143/2286] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:39 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:39.893383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:39.989981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a52ad-b259-4d2d-9650-6db1ab125f45 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.002157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f79b8-885d-4a0d-b1d0-d2cebb4dd6c6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.007805 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.010360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.010690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.011163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.012031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:40.012420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.015345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.035324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1144/2287] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:40.035324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.049038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756175f8-09c6-43f3-9f3c-4271be9a8bb5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.056160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.056877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.057273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.057595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.059177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.062395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.109873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1145/2288] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:40 2022] POST /identity/v3/users => generated 288 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:40.109873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.111581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.115686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.116452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.117237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.125632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc4a0-cae9-4d45-b20f-e8a94ce53a84 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.136397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.137266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.137632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.137911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.138130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.138845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.141709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.142307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.177936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1144/2289] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:40.181181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.218214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1146/2290] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:40 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:40.219589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.231411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b0558-9941-49ce-828c-2ecdb705e31b service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.306087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:40.355966 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218b531-7862-45f6-a1d2-1df124e1cdf5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.374765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.376616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.376616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.376988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.378002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.381202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.420515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:40.446944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1147/2291] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:40 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:40.452126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.460333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.462091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.467803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.468376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.469704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.471845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.484296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81664a-9713-4726-ac90-213aeea3c882 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.499832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.502100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.502655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.503151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.504345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.506349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.523520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1145/2292] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:40.523841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1148/2293] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:40.529170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.530891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.559359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efdd0f-d7f2-4aa1-8c53-1d984499563f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.563329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac839-0646-4cfd-9c28-2920bedbad89 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.574549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.576043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.576695 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.577048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.578317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.580208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.587484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.588381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.588845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.589388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.590784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.593666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.619204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1149/2294] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:40.623332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.632807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1146/2295] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:40.646713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:40.660927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b04e5c-f708-4036-8b05-10c4394a6154 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.689329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.697442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.698670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.699738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] PATH_INFO: `/v3/projects/fd329d42d9a547279b2cf5d7e809f39b/users/33f9c0854b8d437bb58a9dae221b5e31/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.701232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3eb53-f19b-4a8d-af0b-25a5d653d364 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:40.730874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:40.732659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.732927 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.733260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] PATH_INFO: `/v3/projects/1816c9447a324c4099736e9e849ef89e/users/69269a8847354dc29bc70a619ab0860e/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.743350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd329d42d9a547279b2cf5d7e809f39b, user_id=33f9c0854b8d437bb58a9dae221b5e31, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.761981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebf3cb-c79b-46d0-a508-dc2c3f8ea680 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.784061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1816c9447a324c4099736e9e849ef89e, user_id=69269a8847354dc29bc70a619ab0860e, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:40.796892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798aa3a6-05fd-42a8-ab13-0e6ade906092 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:40.803500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1150/2296] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:40 2022] PUT /identity/v3/projects/fd329d42d9a547279b2cf5d7e809f39b/users/33f9c0854b8d437bb58a9dae221b5e31/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:40.821851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.822207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.822450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.875915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1147/2297] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:40 2022] PUT /identity/v3/projects/1816c9447a324c4099736e9e849ef89e/users/69269a8847354dc29bc70a619ab0860e/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:40.900381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:40.914036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:40.918631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:40.946349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] MFA Rules not processed for user `33f9c0854b8d437bb58a9dae221b5e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:40.951668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.024754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] MFA Rules not processed for user `69269a8847354dc29bc70a619ab0860e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:41.027033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.137112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1151/2298] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:41.141009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.141009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.166628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1148/2299] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:41.168202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.311088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e102fe-7d20-4076-9254-1f3ad7d7ceef None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.317722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576104f7-2f89-4139-af78-7ff6ab868903 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.318644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.320149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.320863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.321376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.323011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.326033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.327795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.328284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.328575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.329231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] PATH_INFO: `/v3/projects/ea6c61ae6f344795a0002ab8c5c94094` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.331882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6c61ae6f344795a0002ab8c5c94094)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.336163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.361222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:41.397725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1149/2300] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:41 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:41.397725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.402858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.480331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:41.492475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6c61ae6f344795a0002ab8c5c94094', 'request_id': 'req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:41.519118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6c61ae6f344795a0002ab8c5c94094', 'request_id': 'req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:41.546533 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6c61ae6f344795a0002ab8c5c94094', 'request_id': 'req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:41.581597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] The token cache is being invalidate because project ea6c61ae6f344795a0002ab8c5c94094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:41.581597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:41.582986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:41.587752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1152/2301] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:41 2022] DELETE /identity/v3/projects/ea6c61ae6f344795a0002ab8c5c94094 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:41.592598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.594422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.659287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d1290-f7b4-4a83-8462-8d2ec0cb3f9e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.777691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f222e6-b197-4161-b7dc-a37e33a9d2ba admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.784919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.786096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.786706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.787957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.789154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.792404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.794347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.795044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.795387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.795636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.797846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.842119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1153/2302] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:41 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:41.861031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.878034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.880384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:41.883861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.913124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1150/2303] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:41.914386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:41.918089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73403c0-9e26-4eea-adde-e272eecb7d99 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.940855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695d5b-18af-4f92-99d4-8124ba294ca2 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:41.948913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.950611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.955063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:41.955879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:41.956265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.956551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.957313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.959447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.965596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:41.966559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] PATH_INFO: `/v3/users/6b9db5837ad948ecad8eec7190fe4b49` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:41.971035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9db5837ad948ecad8eec7190fe4b49)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:41.981671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:41.994043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1151/2304] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:41.995649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:42.025461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e92d00-f6b8-49cc-873b-858e1c8a3210 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:42.043563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:42.047607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:42.049188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:42.049696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] PATH_INFO: `/v3/projects/adfdfbad259a42f7a0534f1fb39c326e/users/4f50c355fd7e4ac2845d89d4b469497e/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:42.095851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfdfbad259a42f7a0534f1fb39c326e, user_id=4f50c355fd7e4ac2845d89d4b469497e, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:42.102386 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:42.133819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1152/2305] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:41 2022] PUT /identity/v3/projects/adfdfbad259a42f7a0534f1fb39c326e/users/4f50c355fd7e4ac2845d89d4b469497e/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:42.139562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:42.161013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9db5837ad948ecad8eec7190fe4b49'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:42.205232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9db5837ad948ecad8eec7190fe4b49'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:42.231975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:42.238273 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9db5837ad948ecad8eec7190fe4b49'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:42.282489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9db5837ad948ecad8eec7190fe4b49'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:42.285503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9db5837ad948ecad8eec7190fe4b49'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:42.305128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1154/2306] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:41 2022] DELETE /identity/v3/users/6b9db5837ad948ecad8eec7190fe4b49 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:42.319878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc16060-be7a-4e60-9348-bdcf2dfccc3d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:42.522414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:42.523140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:42.523431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:42.523744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:42.525813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:42.528644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966063e0-226d-446a-89d1-2327c6b0be08 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:42.568007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1153/2307] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:42.569708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:42.609978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:42.732794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c58a4-70ff-4bcd-a093-d47083222fb0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:42.771097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:42.771097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:42.771097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:42.771336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] PATH_INFO: `/v3/projects/adfdfbad259a42f7a0534f1fb39c326e/users/4f50c355fd7e4ac2845d89d4b469497e/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:42.774984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfdfbad259a42f7a0534f1fb39c326e, user_id=4f50c355fd7e4ac2845d89d4b469497e, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:42.786341 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:42.787183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:42.787183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:42.801177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83860a8-98a6-4d06-8d29-2bdadbf2dba9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:42.874788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1155/2308] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:42 2022] PUT /identity/v3/projects/adfdfbad259a42f7a0534f1fb39c326e/users/4f50c355fd7e4ac2845d89d4b469497e/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:42.896616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:42.897113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:42.897623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:42.919554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] MFA Rules not processed for user `15de7bbd869f41baa4e21690c79150dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:42.925677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:42.935037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] MFA Rules not processed for user `4f50c355fd7e4ac2845d89d4b469497e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:42.936346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:42.967927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:43.237451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1154/2309] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:43.316267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1156/2310] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:43.359212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:43.359518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:43.362569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:43.382152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:43.558296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e7bb1-19e8-41f9-a5a0-e45473fa9853 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:43.578683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:43.589147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07ffcd-fc21-4d3a-a2fb-b1c965b41eff None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:43.594699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:43.597104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:43.597465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:43.597919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:43.598206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:43.598506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:43.600514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:43.615018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:43.627527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:43.629917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:43.654770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1157/2311] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:26:43.663346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:43.877847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:43.883616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:43.921598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1155/2312] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:43.929842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:43.948084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0f38-022c-4a25-b9cb-0503cec30c96 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:43.962200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:43.963260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:43.963806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:43.964320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:43.967251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:44.141940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:44.144803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:44.272636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:44.276185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae795368-77ad-451e-af3d-014e33e97083 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:44.341868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1158/2313] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:44.499876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b00a-fe5f-4a5f-bb9b-e24e825d8af6 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:44.524372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:44.527202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:44.529716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:44.529716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] PATH_INFO: `/v3/projects/4955cea805e646ba9b5b211fed96ff81` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:44.537823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4955cea805e646ba9b5b211fed96ff81)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:44.544523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:44.790726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4955cea805e646ba9b5b211fed96ff81', 'request_id': 'req-53029aa5-3884-438f-bafa-d6ac21519f73'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:44.816318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4955cea805e646ba9b5b211fed96ff81', 'request_id': 'req-53029aa5-3884-438f-bafa-d6ac21519f73'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:44.840380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4955cea805e646ba9b5b211fed96ff81', 'request_id': 'req-53029aa5-3884-438f-bafa-d6ac21519f73'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:44.875988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] The token cache is being invalidate because project 4955cea805e646ba9b5b211fed96ff81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:44.878641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:44.880560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-53029aa5-3884-438f-bafa-d6ac21519f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:44.886841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1156/2314] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:44 2022] DELETE /identity/v3/projects/4955cea805e646ba9b5b211fed96ff81 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:44.891491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140968b-8288-46f9-bcec-421916592f18 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:44.892345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140968b-8288-46f9-bcec-421916592f18 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:44.892839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140968b-8288-46f9-bcec-421916592f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:44.946790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1140968b-8288-46f9-bcec-421916592f18 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:44.948765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1140968b-8288-46f9-bcec-421916592f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:45.087198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:45.192153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1159/2315] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:45.204366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:45.204684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:45.204914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:45.267626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:45.273465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:45.315947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1157/2316] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:45.323733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140968b-8288-46f9-bcec-421916592f18 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:45.369829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140968b-8288-46f9-bcec-421916592f18 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:45.406691 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:45.407775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:45.408239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:45.412501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:45.420147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:45.420767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:45.423652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:45.579043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1160/2317] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:45.581869 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:45.609551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd0615-117e-4484-a62e-541d92168dab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:45.622380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:45.624114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:45.624821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:45.625968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:45.627536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:45.632882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a149d-16ce-4098-a0c2-84d820e2fa52 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:45.716865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1158/2318] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:45 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:45.745771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:45.783015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32c3-09c9-49c8-ab24-5949c76dff1b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:45.876302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:45.876812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:45.877321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:45.936514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:45.952945 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] MFA Rules not processed for user `55d7bbda47b64493a840fef8c6c40431`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:45.954077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:46.046663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.047812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.048141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.048474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.049308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:46.053487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:46.086569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1161/2319] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:45 2022] POST /identity/v3/users => generated 287 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:46.094245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.136835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc659497-a143-4053-a8b1-693f8db7d236 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:46.143560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1159/2320] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:46.164506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.166055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.166706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:46.167889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.168455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.169022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.170501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:46.177308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:46.215844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794c790-d7dc-4d44-97ee-2f3307c2e865 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:46.248183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.251024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.251310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1162/2321] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:46.251908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.253230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.253803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.256974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:46.281913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:46.295310 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec1ec6-e0c5-4228-9427-37289b6dbd1a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:46.310725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.312809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.314959 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.315980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.318191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:46.325506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:46.388782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1163/2322] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:46.390902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.400355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:46.407721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1160/2323] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:46 2022] POST /identity/v3/projects => generated 385 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:26:46.407721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.616670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e3b5-a627-4f6a-b3a9-5d266ee95680 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:46.626991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690e9ab-08d8-44e9-95ef-54418b72ebf9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:46.642441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.644768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.645597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.648764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] PATH_INFO: `/v3/projects/c6ee38d422e94adabdb0b42b642d8fd5/users/c0fd95636d6f4405bffb6bb868c90ee1/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.723785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ee38d422e94adabdb0b42b642d8fd5, user_id=c0fd95636d6f4405bffb6bb868c90ee1, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:46.727614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:46.761109 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1161/2324] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:46 2022] PUT /identity/v3/projects/c6ee38d422e94adabdb0b42b642d8fd5/users/c0fd95636d6f4405bffb6bb868c90ee1/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:46.762285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:46.814250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:46.816888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:46.816888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:46.817128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:46.821201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:46.842603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:47.044024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201290f-dff2-4170-a64c-a913ba2378bc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.051444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.055476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203e3f6-d57f-443e-8fd2-48e59365da1e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.069696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.073880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.074501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.075057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.083810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.090454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1164/2325] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:47.091121 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.094079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.115867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.117287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.136361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:47.207247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] MFA Rules not processed for user `c0fd95636d6f4405bffb6bb868c90ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:47.208524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:47.213737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1162/2326] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:46 2022] POST /identity/v3/users => generated 299 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:47.217968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:47.256417 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2763-9e41-41f1-9195-853fba05e467 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.269850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.271691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.272698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.273322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.275982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.281808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.329261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1163/2327] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:47.331517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:47.373949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0ce6f-7458-460e-9e2b-03cf5d8e3d7f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.398997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.401054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.401660 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.402397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] PATH_INFO: `/v3/projects/9ce8951f40384a89815320ea0108e5e8/users/16b7350f73e14634b6dfa6ad3a72d45c/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.430675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1165/2328] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:47.460131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:47.470058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce8951f40384a89815320ea0108e5e8, user_id=16b7350f73e14634b6dfa6ad3a72d45c, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.473709 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.504669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1164/2329] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:47 2022] PUT /identity/v3/projects/9ce8951f40384a89815320ea0108e5e8/users/16b7350f73e14634b6dfa6ad3a72d45c/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:47.511682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:47.641359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5d67-7047-4ab5-965e-c838db85ed97 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.648946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.650037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.650511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.651016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.652084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.654106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.655871 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757dd1d-8020-4ab9-800b-37a37dd5ccf3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.667871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1165/2330] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:47.672784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:47.691208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f697c64-2bcc-4d8d-92c1-cbf9ec7246cc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:47.697683 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.700214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.700808 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.701498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] PATH_INFO: `/v3/projects/9ce8951f40384a89815320ea0108e5e8/users/16b7350f73e14634b6dfa6ad3a72d45c/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.706527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce8951f40384a89815320ea0108e5e8, user_id=16b7350f73e14634b6dfa6ad3a72d45c, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.714531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:47.756929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1166/2331] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:47 2022] PUT /identity/v3/projects/9ce8951f40384a89815320ea0108e5e8/users/16b7350f73e14634b6dfa6ad3a72d45c/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:47.831819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:47.832816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:47.833153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:47.833507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:47.852026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:47.982920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:47.986247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e796b34-f549-4f70-b0bc-fd80f8ea0e86 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:48.006216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1166/2332] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:48.974192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:48.977725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:49.413665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8c2ee-8c7d-42e2-ba85-6079ab865157 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:49.435273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:49.439343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:49.439343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:49.439343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] PATH_INFO: `/v3/users/33f9c0854b8d437bb58a9dae221b5e31` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:49.441706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f9c0854b8d437bb58a9dae221b5e31)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:49.460390 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:49.590184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:49.616829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f9c0854b8d437bb58a9dae221b5e31'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:49.653636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f9c0854b8d437bb58a9dae221b5e31'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:49.656371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f9c0854b8d437bb58a9dae221b5e31'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:49.680282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f9c0854b8d437bb58a9dae221b5e31'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:49.683373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f9c0854b8d437bb58a9dae221b5e31'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:49.782925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1167/2333] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:48 2022] DELETE /identity/v3/users/33f9c0854b8d437bb58a9dae221b5e31 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:49.792008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:49.794523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:49.795286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:49.871853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:49.873211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:49.984169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1167/2334] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:50.020751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:50.023084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:50.315214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13fdfb-6617-457e-8186-cbf123be5b0a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:50.320397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:50.321533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:50.322007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:50.322480 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:50.336101 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:50.338787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:50.354662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1168/2335] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:51.073822 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:51.078403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:51.148321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcace9-4120-4b4b-86aa-0dea23c066c0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:51.173641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:51.177759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:51.180118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:51.181165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] PATH_INFO: `/v3/projects/fd329d42d9a547279b2cf5d7e809f39b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:51.189644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd329d42d9a547279b2cf5d7e809f39b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:51.207190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:51.355152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd329d42d9a547279b2cf5d7e809f39b', 'request_id': 'req-a8757116-1018-475f-ba9f-7a87413187cd'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.387116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd329d42d9a547279b2cf5d7e809f39b', 'request_id': 'req-a8757116-1018-475f-ba9f-7a87413187cd'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.406507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd329d42d9a547279b2cf5d7e809f39b', 'request_id': 'req-a8757116-1018-475f-ba9f-7a87413187cd'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.433907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] The token cache is being invalidate because project fd329d42d9a547279b2cf5d7e809f39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:51.436219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.437466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.440038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1168/2336] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:51 2022] DELETE /identity/v3/projects/fd329d42d9a547279b2cf5d7e809f39b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:51.445853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:51.466008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e775c6-8b0f-432d-862b-8ce7df2670ff service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:51.755326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:51.756350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:51.756636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:51.756967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] PATH_INFO: `/v3/users/4f50c355fd7e4ac2845d89d4b469497e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:51.765839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f50c355fd7e4ac2845d89d4b469497e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:51.767894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:51.872757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f50c355fd7e4ac2845d89d4b469497e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.899894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f50c355fd7e4ac2845d89d4b469497e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.901546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f50c355fd7e4ac2845d89d4b469497e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.924806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f50c355fd7e4ac2845d89d4b469497e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.926977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3b1f76b3-a0ea-4933-b24e-21c6967304e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f50c355fd7e4ac2845d89d4b469497e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:51.951609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:52.084318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1169/2337] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:51 2022] DELETE /identity/v3/users/4f50c355fd7e4ac2845d89d4b469497e => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:53.026372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:53.030901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:53.060430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757116-1018-475f-ba9f-7a87413187cd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:53.181392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:53.292710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:53.294107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.297894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.298603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] PATH_INFO: `/v3/projects/adfdfbad259a42f7a0534f1fb39c326e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.300252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfdfbad259a42f7a0534f1fb39c326e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:53.303008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:53.448470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfdfbad259a42f7a0534f1fb39c326e', 'request_id': 'req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:53.462784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfdfbad259a42f7a0534f1fb39c326e', 'request_id': 'req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:53.481147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfdfbad259a42f7a0534f1fb39c326e', 'request_id': 'req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:53.508483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] The token cache is being invalidate because project adfdfbad259a42f7a0534f1fb39c326e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:53.508786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:53.514891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2f92ac5-343a-4f9d-b61a-547e4ab52c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:53.517175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1169/2338] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:53 2022] DELETE /identity/v3/projects/adfdfbad259a42f7a0534f1fb39c326e => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:53.522871 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.523845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.524160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.557925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:53.559298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:53.723746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1170/2339] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:53.732758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.733247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.733722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.776465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:53.777771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:53.791440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:53.792514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1170/2340] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:53.803483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ee532-f67d-4941-ba80-9c50c34f577c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:53.810207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:53.810902 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.811217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.811439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.812707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:26:53.813210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:53.815429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d2ac2-f7a3-4ab7-9927-38e62bbbad7b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:53.842547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:53.851006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1171/2341] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:26:53.857051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2726b-59d1-40a5-84ec-77b40aa2d77b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:53.862753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:53.865556 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.865770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.865928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.866849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:53.869493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:53.959509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1171/2342] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:53 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:53.960497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:53.960845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:53.961133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:53.965600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:54.005329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] MFA Rules not processed for user `2d1eb89923784f1c83d2077c6b6926e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:54.014942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:54.169393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1172/2343] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:54.183772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3fed-cdb1-4d52-ab5d-fd61dc72aafc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:54.191774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:54.192839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:54.193127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:54.193461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:54.194365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:54.196786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:54.233899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1172/2344] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:53 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:54.235840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:54.261595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d18c3c-a160-4fe0-8243-909f9dd96d9c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:54.271073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:54.272343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:54.272926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:54.273558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:54.275364 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:54.279154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:54.312863 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1173/2345] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:26:54.314027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:54.328901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceab36-2064-491a-9292-2c356a3d4a19 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:54.337303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:54.338039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:54.338323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:54.338652 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:54.339407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:54.341413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156da4f7-116c-4513-b330-745ba8befc11 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:54.411672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1173/2346] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:54.412731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:54.428723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2521174-210b-44d8-9f8e-473284adc354 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:54.455455 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:54.456900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:54.457422 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:54.458361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] PATH_INFO: `/v3/projects/ec1736a4aa38419fa767630130009254/users/bdc7b38609cb41daafc24c69f6ccef7e/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:54.487846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1736a4aa38419fa767630130009254, user_id=bdc7b38609cb41daafc24c69f6ccef7e, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:54.491461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:54.558575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1174/2347] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:54 2022] PUT /identity/v3/projects/ec1736a4aa38419fa767630130009254/users/bdc7b38609cb41daafc24c69f6ccef7e/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:54.573091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:54.574245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:54.575171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:54.692444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:54.728003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] MFA Rules not processed for user `bdc7b38609cb41daafc24c69f6ccef7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:54.729804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:54.921838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1174/2348] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:54.927771 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:55.056987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:55.127332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b0f28-b72b-4180-8999-93e3a884be4e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:55.145173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:55.145918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:55.146269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:55.146637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:55.147770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:55.150633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:55.225240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1175/2349] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:26:54 2022] POST /identity/v3/projects => generated 387 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:55.230804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:55.254610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc8fac-3962-4f57-a6cc-c58f9b119316 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:55.435708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:55.436965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:55.437925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:55.438384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:55.439326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:55.446933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:55.504489 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1175/2350] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:26:55 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:26:55.509687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:55.532894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970643f-2377-4619-865b-2c2300ee74ad admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:55.552346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:55.553564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:55.554032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:55.554537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:55.555618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:55.557844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:55.579222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1176/2351] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:55.585510 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:55.601962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6896187-0852-4c8c-92c8-f1a57601b23e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:55.616302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:55.617471 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:55.617764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:55.618045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] PATH_INFO: `/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef/users/22934fc2f9b3425ab541ce2b73ea24f2/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:55.676835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c52c7ae7b2a4220a79b4225b68b0aef, user_id=22934fc2f9b3425ab541ce2b73ea24f2, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:55.684621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:55.728486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1176/2352] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:55 2022] PUT /identity/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef/users/22934fc2f9b3425ab541ce2b73ea24f2/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:26:55.733600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:55.788127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbebe5-bbae-4ddb-9a34-574f4e144bc6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:55.924948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:55.926049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:55.926472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:55.926905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:55.928096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:55.930920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0f77c-5362-42b6-93ac-a828d83a1cfa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:55.961165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1177/2353] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:26:55.967546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:56.014878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c2079-d697-4d86-9f7f-f65f510cfef5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:56.034913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:56.037389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.037614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.041109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] PATH_INFO: `/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef/users/22934fc2f9b3425ab541ce2b73ea24f2/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:56.045230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c52c7ae7b2a4220a79b4225b68b0aef, user_id=22934fc2f9b3425ab541ce2b73ea24f2, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:56.052481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:56.090815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1177/2354] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:26:55 2022] PUT /identity/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef/users/22934fc2f9b3425ab541ce2b73ea24f2/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:56.094318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.094799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.095328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:56.187523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] MFA Rules not processed for user `22934fc2f9b3425ab541ce2b73ea24f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:56.192043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:56.303811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:56.307378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:56.444725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1178/2355] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:56.453283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:56.455185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:56.512141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:56.550284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022b598-521d-449a-872b-3925bab73b8e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:56.562135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:56.572986 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:56.576271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.576783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.577281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] PATH_INFO: `/v3/users/c0fd95636d6f4405bffb6bb868c90ee1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:56.583951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0fd95636d6f4405bffb6bb868c90ee1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:56.592175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:56.610634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f7520-02f5-4699-bf71-41cd291eb5c2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:56.620540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:56.621678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.622120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.622538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:56.657061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:56.662506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372df3-a4e1-4990-b769-f2619f2da4fe service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:56.700224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1179/2356] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:56.790774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0fd95636d6f4405bffb6bb868c90ee1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:56.813764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0fd95636d6f4405bffb6bb868c90ee1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:56.839214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0fd95636d6f4405bffb6bb868c90ee1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:56.856610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.857783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.858226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:56.891100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0fd95636d6f4405bffb6bb868c90ee1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:56.894509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d9971ac5-4807-4e28-a46e-2a72ab51b615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0fd95636d6f4405bffb6bb868c90ee1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:56.908934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:56.916034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:56.920313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1178/2357] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:56 2022] DELETE /identity/v3/users/c0fd95636d6f4405bffb6bb868c90ee1 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:56.925909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:56.926552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:56.927158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:57.002164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:57.004873 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:57.165804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1180/2358] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:26:57.282242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1179/2359] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:57.304653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:57.308382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:57.364641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:57.369557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:57.557259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f3ae6-027d-4e6c-84ba-72cef1277746 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:57.572895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:57.578977 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:57.580267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:57.581455 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:57.623840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92814e08-7d2a-4294-9592-cbff37e3cba6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:57.631769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:57.644259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:57.646203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:57.649570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:57.650220 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:57.650723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:57.677737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1181/2360] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:57.679641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:57.679826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:57.681961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3abc1-4448-4efa-ba0b-b41e9c486b1c service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:57.682193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:57.706221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1180/2361] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:26:57.735200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123ed4a-1852-4166-9057-74e562bf90b3 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:57.751194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:57.755608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:57.756135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:57.756572 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] PATH_INFO: `/v3/users/bdc7b38609cb41daafc24c69f6ccef7e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:57.763082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc7b38609cb41daafc24c69f6ccef7e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:57.765318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:58.037302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc7b38609cb41daafc24c69f6ccef7e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:58.077361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc7b38609cb41daafc24c69f6ccef7e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:58.079063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc7b38609cb41daafc24c69f6ccef7e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:58.119596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc7b38609cb41daafc24c69f6ccef7e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:58.122255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc7b38609cb41daafc24c69f6ccef7e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:58.185300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:58.223335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1182/2362] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:57 2022] DELETE /identity/v3/users/bdc7b38609cb41daafc24c69f6ccef7e => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:58.229639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:58.230298 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:58.231067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:58.245504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:26:58.277544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:58.282067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:58.494659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1181/2363] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:58.550842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:58.553967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:58.722858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030696d1-6fd8-4477-8da4-7f26930ea195 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:58.728284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:58.729319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:58.729804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:58.730393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:58.742804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:58.744443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d033ad-0731-41d2-a1fe-7f3d4ce8525a service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:58.757098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1183/2364] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:26:59.072844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:59.074843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:59.100244 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2deda-10ed-4733-a031-353de0113c30 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:59.116570 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:59.118332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:59.118973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:59.119767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] PATH_INFO: `/v3/projects/c6ee38d422e94adabdb0b42b642d8fd5` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:59.128688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ee38d422e94adabdb0b42b642d8fd5)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:59.131167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:26:59.264640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ee38d422e94adabdb0b42b642d8fd5', 'request_id': 'req-219634f3-098f-46a4-bc8f-170f21f2577f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:59.305599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ee38d422e94adabdb0b42b642d8fd5', 'request_id': 'req-219634f3-098f-46a4-bc8f-170f21f2577f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:59.371158 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ee38d422e94adabdb0b42b642d8fd5', 'request_id': 'req-219634f3-098f-46a4-bc8f-170f21f2577f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:59.475902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] The token cache is being invalidate because project c6ee38d422e94adabdb0b42b642d8fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:26:59.476416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:59.485441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-219634f3-098f-46a4-bc8f-170f21f2577f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:26:59.493307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1182/2365] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:26:59 2022] DELETE /identity/v3/projects/c6ee38d422e94adabdb0b42b642d8fd5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:26:59.498748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:59.499006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:59.499124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:59.555743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:59.556981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:59.860097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1184/2366] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:26:59.860852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:59.861247 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:59.861505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:59.908567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:26:59.910880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:59.946231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c0627-416e-44bc-8853-c98801bdbee6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:26:59.948665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:26:59.950693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:26:59.960868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:26:59.962129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:26:59.962592 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:26:59.962978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] PATH_INFO: `/v3/users/69269a8847354dc29bc70a619ab0860e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:26:59.968184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69269a8847354dc29bc70a619ab0860e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:26:59.995013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:00.000399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1183/2367] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:00.007303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:00.062133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e04a1c-cf56-4440-a965-be01fe54b7ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:00.110212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:00.149954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:00.152825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:00.154882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:00.164567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:00.165171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:00.176357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:00.186258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69269a8847354dc29bc70a619ab0860e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:00.211166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69269a8847354dc29bc70a619ab0860e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:00.211166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69269a8847354dc29bc70a619ab0860e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:00.229701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69269a8847354dc29bc70a619ab0860e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:00.230695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1d5743d1-a6a0-41b6-b048-f77c1d906d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69269a8847354dc29bc70a619ab0860e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:00.299624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1185/2368] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:26:59 2022] DELETE /identity/v3/users/69269a8847354dc29bc70a619ab0860e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:00.302813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:00.303613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:00.304752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:00.313550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1184/2369] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:00.320360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:00.325018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:00.381440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:00.386646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:00.401723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f754-3a59-4953-9c34-824d1c578099 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:00.424765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:00.689755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:00.691087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:00.691383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:00.691664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:00.692779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:00.694976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:00.709527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1186/2370] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:00.714424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:00.750061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:00.775693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:00.827106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033d7e8-0967-4bf4-9f42-4b7ef23d5a02 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:00.862134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:00.870347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:00.874921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1185/2371] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:00 2022] POST /identity/v3/projects => generated 381 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:00.877011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:00.884830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] PATH_INFO: `/v3/projects/ec1736a4aa38419fa767630130009254` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:00.885413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:00.896132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:00.910459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1736a4aa38419fa767630130009254)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:00.923161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.120880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1736a4aa38419fa767630130009254', 'request_id': 'req-6c9464a7-de62-4df3-8386-a714271f1025'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.130610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1736a4aa38419fa767630130009254', 'request_id': 'req-6c9464a7-de62-4df3-8386-a714271f1025'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.139117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1736a4aa38419fa767630130009254', 'request_id': 'req-6c9464a7-de62-4df3-8386-a714271f1025'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.180720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] The token cache is being invalidate because project ec1736a4aa38419fa767630130009254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:01.182750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.189007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.193748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1187/2372] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:00 2022] DELETE /identity/v3/projects/ec1736a4aa38419fa767630130009254 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:01.195777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:01.202962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:01.216434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145bf05-9c2d-48b1-ab98-4f80dfc5c9d2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.378523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.379377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.379705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.379988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.385457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:01.403993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9464a7-de62-4df3-8386-a714271f1025 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.411515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.412860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.413173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.413512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.414264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.420186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.422878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.435014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.438328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1186/2373] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:01.439549 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:01.444381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:01.487025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e14bb-55e0-4b0f-8bc2-0f4d2402f171 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.499901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.502973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.503591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.504033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] PATH_INFO: `/v3/users/22934fc2f9b3425ab541ce2b73ea24f2` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.508947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1188/2374] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:01 2022] POST /identity/v3/users => generated 291 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:01.509627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] RBAC: Authorizing `identity:delete_user(user_id=22934fc2f9b3425ab541ce2b73ea24f2)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.509985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:01.514447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.580471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74489b1c-853d-4dae-bdee-4a7c8702d9c5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.593040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.595759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.596932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.598809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.606035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.613648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.640634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1189/2375] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:01.644938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:01.646423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22934fc2f9b3425ab541ce2b73ea24f2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.666226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb109-1b07-437b-a7be-0d98d913cf1c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.685038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.685483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22934fc2f9b3425ab541ce2b73ea24f2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.687546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.687855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.688270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.692872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.699661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.701557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22934fc2f9b3425ab541ce2b73ea24f2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.734895 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1190/2376] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:27:01.740529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22934fc2f9b3425ab541ce2b73ea24f2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.741497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:01.743316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-27fc5a59-6954-4266-93db-332300c6423d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22934fc2f9b3425ab541ce2b73ea24f2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:01.764128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:01.766972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1187/2377] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:01 2022] DELETE /identity/v3/users/22934fc2f9b3425ab541ce2b73ea24f2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:01.772022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd496626-ec3f-40e3-93ff-eb9e6b154755 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:01.874478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:01.876336 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.876747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.877322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] PATH_INFO: `/v3/projects/bd9b5f7127564335a847ab40761124df/users/c7e1898925cd4feea900627bc0ee8a7a/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:01.903037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9b5f7127564335a847ab40761124df, user_id=c7e1898925cd4feea900627bc0ee8a7a, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:01.906611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:01.948781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1191/2378] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:01 2022] PUT /identity/v3/projects/bd9b5f7127564335a847ab40761124df/users/c7e1898925cd4feea900627bc0ee8a7a/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:01.954558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:01.955411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:01.955718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:02.042416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:02.061771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:02.190260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:02.388076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1188/2379] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:02.479171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:02.481940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:02.682518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70676715-a2b6-47fb-8ee0-245a1b40b575 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:02.699043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:02.699739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:02.700019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:02.700513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:02.727380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:02.729587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08840940-5a01-4b05-bb74-fd9b09a09c9e service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:02.768350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1192/2380] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:27:03.725406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:03.729015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:03.758749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4c926-c357-4706-bb2a-9e45abfe533e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:03.775875 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:03.777254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:03.780494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:03.781164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] PATH_INFO: `/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:03.785860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c52c7ae7b2a4220a79b4225b68b0aef)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:03.792093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:03.914535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2212b-bd37-4ee9-8fb1-111b547fee2a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:03.915069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2212b-bd37-4ee9-8fb1-111b547fee2a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:03.915223 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2212b-bd37-4ee9-8fb1-111b547fee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:03.973545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c52c7ae7b2a4220a79b4225b68b0aef', 'request_id': 'req-c937eb58-52ce-4855-aab9-dd639a3daff6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.005978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c52c7ae7b2a4220a79b4225b68b0aef', 'request_id': 'req-c937eb58-52ce-4855-aab9-dd639a3daff6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.020008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c52c7ae7b2a4220a79b4225b68b0aef', 'request_id': 'req-c937eb58-52ce-4855-aab9-dd639a3daff6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.040113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] The token cache is being invalidate because project 9c52c7ae7b2a4220a79b4225b68b0aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:04.040888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.042133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.049407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1189/2381] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:03 2022] DELETE /identity/v3/projects/9c52c7ae7b2a4220a79b4225b68b0aef => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:04.050827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:04.052726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.062098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-87b2212b-bd37-4ee9-8fb1-111b547fee2a None None] MFA Rules not processed for user `16b7350f73e14634b6dfa6ad3a72d45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:04.065134 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-87b2212b-bd37-4ee9-8fb1-111b547fee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.174687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937eb58-52ce-4855-aab9-dd639a3daff6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:04.181302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:04.182987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:04.184106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:04.184994 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] PATH_INFO: `/v3/projects/1816c9447a324c4099736e9e849ef89e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:04.187398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1816c9447a324c4099736e9e849ef89e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:04.201212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:04.349153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:04.351938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1816c9447a324c4099736e9e849ef89e', 'request_id': 'req-47f2a740-737e-4afe-9975-cd4f37c0cdb6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.372121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1816c9447a324c4099736e9e849ef89e', 'request_id': 'req-47f2a740-737e-4afe-9975-cd4f37c0cdb6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.385095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1193/2382] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:04.385458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1816c9447a324c4099736e9e849ef89e', 'request_id': 'req-47f2a740-737e-4afe-9975-cd4f37c0cdb6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.386365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f3243-55c3-40b9-924c-aae287adfb1c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:04.386890 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f3243-55c3-40b9-924c-aae287adfb1c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:04.387371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f3243-55c3-40b9-924c-aae287adfb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:04.414863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] The token cache is being invalidate because project 1816c9447a324c4099736e9e849ef89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:04.416523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.417532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:04.422965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1190/2383] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:04 2022] DELETE /identity/v3/projects/1816c9447a324c4099736e9e849ef89e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:04.424479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:04.428475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.496561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5a0f3243-55c3-40b9-924c-aae287adfb1c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:04.499160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a0f3243-55c3-40b9-924c-aae287adfb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.544546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:04.689910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2a740-737e-4afe-9975-cd4f37c0cdb6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:04.700702 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:04.709813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:04.710191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:04.710832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:04.717823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.868785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1194/2384] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:04.877884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:04.878149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:04.878463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:04.954701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:04.956087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:04.988746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1195/2385] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:04.990087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.010858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.017912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.028949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639290fd-4e5b-4a8f-84b9-b20416d3ae63 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.056594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.062843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.063721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.066698 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.067885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:05.073432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.075668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.084586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1191/2386] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:05.144627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1196/2387] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:05.147513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.261192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47361-4633-4e04-ab64-3b52b0f03db8 service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.280738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.282228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.282768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.283263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.285554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.292304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fcd82-6cda-407f-a90e-59c50c85587a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.420819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.433594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1192/2388] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:05 2022] POST /identity/v3/projects => generated 385 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:05.440860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7edd-17be-42e5-9ff5-d4385a0be189 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.607164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.607666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.608125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.639737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.640444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.640727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.641007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.642210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.646266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.649264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:05.651155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:05.688020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1197/2389] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:05 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:05.702525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.724209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e117f9-1612-4546-8270-031ae8801679 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.731762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.733135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.733855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.734369 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.735321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.737852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.764820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:05.793270 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1198/2390] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:05.799096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.809778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1193/2391] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:27:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:05.815837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e1755-71b5-4d08-9638-58b574503238 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.823382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.824842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:05.848256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.849935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.851096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.853254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:05.857295 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:05.868240 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:05.904032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1199/2392] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:05.909274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:05.940850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab8487-f22f-4cd8-a55c-a8c772119cb6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:05.961021 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:05.963275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:05.964170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:05.964807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] PATH_INFO: `/v3/projects/cd370488060e4d629e27fb51150257a4/users/837d7f9e0adf4348953791ae814e4372/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:06.016290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd370488060e4d629e27fb51150257a4, user_id=837d7f9e0adf4348953791ae814e4372, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:06.019787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1504609-3d61-44d7-b262-a461b4838bd9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:06.053664 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1200/2393] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:05 2022] PUT /identity/v3/projects/cd370488060e4d629e27fb51150257a4/users/837d7f9e0adf4348953791ae814e4372/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:06.055742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd7826-0a83-418c-b8d6-3d251f0a340c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:06.064484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:06.064840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:06.065046 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:06.107231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] MFA Rules not processed for user `837d7f9e0adf4348953791ae814e4372`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:06.108616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:06.179149 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:06.253550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1201/2394] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:06.264451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:06.268782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:06.289912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:06.293671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:06.295095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:06.296880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:06.452979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11784822-1371-4c7c-b0aa-74b2569a07a9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:06.468762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:06.470522 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:06.471212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:06.471907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:06.516253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:06.530824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:06.605480 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:06.610068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1202/2395] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:06.620399 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c323375-3ca6-4b2a-8f31-c3c549ead197 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:06.671130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:06.715324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1194/2396] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:27:07.054204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aa67e-f6e9-4c5d-96bc-19cfca9d8f18 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:07.079818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:07.081256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:07.081776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:07.082266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] PATH_INFO: `/v3/users/837d7f9e0adf4348953791ae814e4372` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:07.108182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=837d7f9e0adf4348953791ae814e4372)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:07.138440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:07.399148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837d7f9e0adf4348953791ae814e4372'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:07.425704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837d7f9e0adf4348953791ae814e4372'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:07.434203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837d7f9e0adf4348953791ae814e4372'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:07.445385 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837d7f9e0adf4348953791ae814e4372'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:07.446269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837d7f9e0adf4348953791ae814e4372'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:07.479157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:07.535261 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1203/2397] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:06 2022] DELETE /identity/v3/users/837d7f9e0adf4348953791ae814e4372 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:07.538482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66006b70-61d7-4161-a6cb-470378da15af None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:07.539237 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66006b70-61d7-4161-a6cb-470378da15af None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:07.539407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66006b70-61d7-4161-a6cb-470378da15af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:07.671575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-66006b70-61d7-4161-a6cb-470378da15af None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:07.681715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-66006b70-61d7-4161-a6cb-470378da15af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:07.871906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:07.987032 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1195/2398] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:08.215711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:08.220180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:08.645699 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfdc36-e4e9-4b2b-ae9d-b6834bcbcd3a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:08.765437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:08.770434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:08.771211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:08.776034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:08.817093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:08.830300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d323cb-2397-4f14-a277-549ad2db3f36 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:08.886357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1204/2399] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:11.603774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66006b70-61d7-4161-a6cb-470378da15af None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:11.605638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-66006b70-61d7-4161-a6cb-470378da15af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:11.631212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66006b70-61d7-4161-a6cb-470378da15af None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:11.646544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:11.651249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:11.651582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:11.653743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] PATH_INFO: `/v3/projects/cd370488060e4d629e27fb51150257a4` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:11.658762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd370488060e4d629e27fb51150257a4)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:11.661608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:11.885434 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd370488060e4d629e27fb51150257a4', 'request_id': 'req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:11.906710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd370488060e4d629e27fb51150257a4', 'request_id': 'req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:11.928437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd370488060e4d629e27fb51150257a4', 'request_id': 'req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:11.996558 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] The token cache is being invalidate because project cd370488060e4d629e27fb51150257a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:11.997019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:11.998192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04f8448a-ba5b-4b64-b3e0-8dc251bbdcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:12.010786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5cb64-0b23-4ac6-9424-b388e37d6170 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.011674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5cb64-0b23-4ac6-9424-b388e37d6170 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.011931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5cb64-0b23-4ac6-9424-b388e37d6170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.017279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1196/2400] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:11 2022] DELETE /identity/v3/projects/cd370488060e4d629e27fb51150257a4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:12.094383 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.095145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.095745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.130751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-45c5cb64-0b23-4ac6-9424-b388e37d6170 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:12.134882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-45c5cb64-0b23-4ac6-9424-b388e37d6170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:12.194438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] MFA Rules not processed for user `c7e1898925cd4feea900627bc0ee8a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:12.200409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:12.547093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1205/2401] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:12.549658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.550114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.550573 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.569944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1197/2402] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:12.596093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:12.599243 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:12.648857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:12.655903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:12.717937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1206/2403] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:12.720828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:12.776286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc876f21-acea-40db-aac4-f74d4715202a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:12.817822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:12.821568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.822044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.822538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.830275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:12.830861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:12.838056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:12.890416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1207/2404] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:12.898011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:12.906232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611cb82-4af2-477d-9d8c-c0597c00d069 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:12.925876 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:12.926987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.927451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.927913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.952080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd1e89-1345-48a4-9b68-44258440e51e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:12.963575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:12.973136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:12.975151 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:12.976352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:12.976805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:12.977225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:12.980242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:12.981099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:12.993799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:13.009889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1198/2405] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:27:13.129175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:13.129458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1208/2406] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:12 2022] POST /identity/v3/projects => generated 377 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:13.242630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eef922-da4e-4db8-87bc-468c7a52bc1a service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:13.774638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:13.777292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:13.779227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:13.780319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:13.782733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:13.787900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:13.897997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:13.899686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1199/2407] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:13 2022] POST /identity/v3/users => generated 289 bytes in 767 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:13.924478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b18b-f2ee-4d46-b370-4d6fde310ee6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:13.955107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:13.956376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:13.956841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:13.957315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:13.965678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:13.969830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:14.040994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1209/2408] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:14.043124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:14.094019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9288-e916-4a84-94e0-87010e585ace admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:14.117532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:14.119522 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:14.119710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:14.120151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:14.121615 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:14.132505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:14.196849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1200/2409] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:14.203393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:14.228370 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c028d-60b2-47dd-a01d-58c657b12f65 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:14.251731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:14.253753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:14.254498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:14.255351 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] PATH_INFO: `/v3/projects/f5573c4a7dfc4162950105f0a538881c/users/003f66696906480780203ab01fbcc519/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:14.321453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5573c4a7dfc4162950105f0a538881c, user_id=003f66696906480780203ab01fbcc519, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:14.327621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:14.331261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:14.333957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4b29-a3fe-4e32-9991-82cfbdc7a8a1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:14.383926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1210/2410] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:14 2022] PUT /identity/v3/projects/f5573c4a7dfc4162950105f0a538881c/users/003f66696906480780203ab01fbcc519/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:14.406168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:14.406984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:14.408988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fea58b-28b9-4272-96d3-3d86d31fc0b2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:14.409821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:14.559353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] MFA Rules not processed for user `003f66696906480780203ab01fbcc519`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:14.564219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:14.607929 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:14.785309 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:14.791701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:14.794713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:14.796207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] PATH_INFO: `/v3/users/871dd9685c5e45a88a4c6e8be052f686` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:14.802524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871dd9685c5e45a88a4c6e8be052f686)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:14.817705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:14.972431 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1211/2411] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:15.002392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871dd9685c5e45a88a4c6e8be052f686'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:15.023846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871dd9685c5e45a88a4c6e8be052f686'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:15.031374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:15.051205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871dd9685c5e45a88a4c6e8be052f686'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:15.080649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:15.100884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871dd9685c5e45a88a4c6e8be052f686'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:15.102063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9720c2b4-2c6e-45ef-b589-c4a69dd98666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871dd9685c5e45a88a4c6e8be052f686'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:15.134914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1201/2412] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:14 2022] DELETE /identity/v3/users/871dd9685c5e45a88a4c6e8be052f686 => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:15.323523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989a39-b54d-49c1-a7b8-3e7272811f1d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:15.553772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:15.554917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:15.555607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:15.585828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:15.586964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:15.593543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:15.593797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:15.598283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] MFA Rules not processed for user `c7e1898925cd4feea900627bc0ee8a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:15.600495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:15.605087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:15.809065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1202/2413] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:15.814661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:15.822193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:16.000918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:16.002846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:16.044387 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1212/2414] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 1012 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:27:16.205444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4ad7-7556-4d67-a5a0-5e00b4d926ba None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:16.244984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:16.253019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:16.253609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:16.254532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] PATH_INFO: `/v3/projects/4318b5b98e084c678ce43a44e8fefccb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:16.256944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4318b5b98e084c678ce43a44e8fefccb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:16.264985 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:16.425065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4318b5b98e084c678ce43a44e8fefccb', 'request_id': 'req-7f537af0-47f2-4b90-8235-35fe9f79a9db'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:16.445094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4318b5b98e084c678ce43a44e8fefccb', 'request_id': 'req-7f537af0-47f2-4b90-8235-35fe9f79a9db'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:16.459704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4318b5b98e084c678ce43a44e8fefccb', 'request_id': 'req-7f537af0-47f2-4b90-8235-35fe9f79a9db'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:16.507454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] The token cache is being invalidate because project 4318b5b98e084c678ce43a44e8fefccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:16.513466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:16.517863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:16.529593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1203/2415] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:15 2022] DELETE /identity/v3/projects/4318b5b98e084c678ce43a44e8fefccb => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:16.537113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:16.560215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48d457-b48d-4f8e-ba76-c5d073fd32a7 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:16.581428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:16.762362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:16.764195 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:16.764714 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:16.765226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] PATH_INFO: `/v3/users/15de7bbd869f41baa4e21690c79150dc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:16.783086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15de7bbd869f41baa4e21690c79150dc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:16.791615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:16.994755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15de7bbd869f41baa4e21690c79150dc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:17.032708 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15de7bbd869f41baa4e21690c79150dc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:17.036380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15de7bbd869f41baa4e21690c79150dc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:17.059461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15de7bbd869f41baa4e21690c79150dc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:17.062286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-36460bc6-4e9c-4538-8744-4374604b9f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15de7bbd869f41baa4e21690c79150dc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:17.170917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1213/2416] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:16 2022] DELETE /identity/v3/users/15de7bbd869f41baa4e21690c79150dc => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:17.982436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:17.984884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:18.063756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537af0-47f2-4b90-8235-35fe9f79a9db admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:18.187609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:18.286700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:18.286700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:18.287270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:18.288461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] PATH_INFO: `/v3/projects/2c6c1bc01f754751a8330d6cf311a30f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:18.291175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6c1bc01f754751a8330d6cf311a30f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:18.294262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:18.495667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6c1bc01f754751a8330d6cf311a30f', 'request_id': 'req-b4d74f9b-581e-4ff2-bf04-96207eb006c0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:18.525954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6c1bc01f754751a8330d6cf311a30f', 'request_id': 'req-b4d74f9b-581e-4ff2-bf04-96207eb006c0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:18.543900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6c1bc01f754751a8330d6cf311a30f', 'request_id': 'req-b4d74f9b-581e-4ff2-bf04-96207eb006c0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:18.593275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] The token cache is being invalidate because project 2c6c1bc01f754751a8330d6cf311a30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:18.593775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:18.595070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b4d74f9b-581e-4ff2-bf04-96207eb006c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:18.604097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1204/2417] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:17 2022] DELETE /identity/v3/projects/2c6c1bc01f754751a8330d6cf311a30f => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:18.606275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:18.607099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:18.607099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:18.665616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:18.670124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:18.814190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1214/2418] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:18.817002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:18.817681 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:18.818161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:18.913580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:18.915669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:18.955930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1205/2419] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:18.956703 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:18.972122 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92f1e-c82a-4be4-a77c-b131dabd1169 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:18.984211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:18.986186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:18.987352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:18.991010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:18.991276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:18.991626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:19.001924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:19.064533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1215/2420] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:19.067892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:19.122400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5e358-bc66-4508-bcd4-d9ccf3e1e448 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:19.162988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:19.164118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:19.164602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:19.165069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:19.166484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:19.171118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:19.303210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1206/2421] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:19 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:19.305650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:19.350512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c37f-642a-438a-ae72-098e19aebd53 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:19.504470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:19.695903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:19.703281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:19.704349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:19.705559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:19.709204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:19.716605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:19.806537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1216/2422] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:19 2022] POST /identity/v3/users => generated 288 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:19.812339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:19.857083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11e47-ecc1-44ca-8dad-756fb9d9dbd9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:19.873898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:19.875626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:19.876116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:19.876520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:19.882255 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:19.888855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:19.936017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1207/2423] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:19.942432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:19.970115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5a53b-31be-4e70-bc89-a98572cb2ae3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:19.979540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:19.981502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:19.982268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:19.982993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:19.984556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:19.988180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313a3a1-1b3c-4e56-973e-dca20be5908a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:20.026556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1217/2424] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:20.032557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:20.050928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28b2-3fdb-456a-9ff7-7b23adc06478 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:20.071678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:20.073642 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:20.074044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:20.074602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] PATH_INFO: `/v3/projects/901cfd316e24473180a363e7974b2ed8/users/6205a3afef3c4f219afa9451ab5a24e3/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:20.112013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901cfd316e24473180a363e7974b2ed8, user_id=6205a3afef3c4f219afa9451ab5a24e3, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:20.118770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:20.168103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1208/2425] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:20 2022] PUT /identity/v3/projects/901cfd316e24473180a363e7974b2ed8/users/6205a3afef3c4f219afa9451ab5a24e3/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:20.173292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e4f6-fa80-46be-97bf-49dfca401fe3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:20.173635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e4f6-fa80-46be-97bf-49dfca401fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:20.174185 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e4f6-fa80-46be-97bf-49dfca401fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:20.256969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e188e4f6-fa80-46be-97bf-49dfca401fe3 None None] MFA Rules not processed for user `6205a3afef3c4f219afa9451ab5a24e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:20.259673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e188e4f6-fa80-46be-97bf-49dfca401fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:20.419586 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1218/2426] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:27.334089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:27.339752 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:27.401482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:27.403176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:27.403870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:27.471481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] MFA Rules not processed for user `55d7bbda47b64493a840fef8c6c40431`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:27.472911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:27.534371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca2ac-6c0d-4d7d-a359-516badd692d6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:27.538683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:27.539464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:27.539775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:27.540059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:27.548040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:27.551025 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:27.552932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:27.567331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1209/2427] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:27.657526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1219/2428] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:31.278853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:31.280677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:31.347690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:31.350960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:31.463688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:31.615938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c088aa-14d8-4c65-b952-1434d5688f28 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:31.633441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:31.636340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:31.641943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:31.643735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:31.648399 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:31.650050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4e3ec-4e65-463c-995d-ddb395a4819d service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:31.674671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:31.677585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:31.684832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:31.685539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:31.700379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:31.702935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e116ce-37a9-435f-bbdd-032c1e7587f0 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:31.717378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:31.720626 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:31.759397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1220/2429] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:31.782082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1210/2430] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:36.281202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:36.282114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:36.282114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:36.312962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:36.314332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:36.395310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1221/2431] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:27:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:36.428737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:36.430862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:36.532690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e5620-bd5e-4db3-8aad-1600e1d9cd6e service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:36.538470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:36.539150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:36.539431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:36.539703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:36.549487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:36.551204 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05d14e-7b64-4f0f-83f6-0574f2768251 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:36.560716 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1211/2432] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:27:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:27:42.417665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:42.419797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:42.547268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357d4ab-0cee-49d3-8a52-0c6c65ae9f94 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:42.556071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:42.567329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:42.567329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:42.567329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] PATH_INFO: `/v3/users/003f66696906480780203ab01fbcc519` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:42.567329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003f66696906480780203ab01fbcc519)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:42.567329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:42.607836 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:42.608784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:42.608784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:42.639589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:42.641197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:42.666529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003f66696906480780203ab01fbcc519'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:42.670887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1212/2433] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:27:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:42.685080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003f66696906480780203ab01fbcc519'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:42.686137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003f66696906480780203ab01fbcc519'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:42.706275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:42.708129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003f66696906480780203ab01fbcc519'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:42.709650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8878abd-b87e-4c95-8242-b34a19ac5912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003f66696906480780203ab01fbcc519'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:42.727313 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d47445-8bc8-468f-8729-2f008ee7e5c3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:42.732903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:42.739483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:42.740931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:42.742156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:42.743282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:42.747105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:42.765384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1222/2434] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:42 2022] DELETE /identity/v3/users/003f66696906480780203ab01fbcc519 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:42.769254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:42.769849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:42.770420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:42.773121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:42.776566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:42.795306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1213/2435] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:27:42.807524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:42.809043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:42.951325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1223/2436] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:42.963513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:42.967857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:43.066219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbda14c-74a5-4af3-9571-430b9ad723f1 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:43.080911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:43.164591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:43.166092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:43.166687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:43.167300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:43.192972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:43.195521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ac0dc-9202-4769-aa69-eec72516b354 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:43.215635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1214/2437] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:27:44.218525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:44.221545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:44.247161 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2180cc-0e83-4309-8763-a2b7590e9a74 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:44.274320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:44.276036 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:44.276523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:44.277074 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] PATH_INFO: `/v3/projects/f5573c4a7dfc4162950105f0a538881c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:44.279665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5573c4a7dfc4162950105f0a538881c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:44.284232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:44.397570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5573c4a7dfc4162950105f0a538881c', 'request_id': 'req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:44.426779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5573c4a7dfc4162950105f0a538881c', 'request_id': 'req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:44.441233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5573c4a7dfc4162950105f0a538881c', 'request_id': 'req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:44.472299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] The token cache is being invalidate because project f5573c4a7dfc4162950105f0a538881c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:44.472299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:44.472299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bbbe0271-a0ae-4b23-b66b-8c843b39b90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:44.478173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1224/2438] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:44 2022] DELETE /identity/v3/projects/f5573c4a7dfc4162950105f0a538881c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:44.480006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:44.480830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:44.481498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:44.536476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:44.539242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:44.664353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1215/2439] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:44.684504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:44.684999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:44.685452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:44.738410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:44.742371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:44.761649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1225/2440] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:44.768195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:44.800982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b53b6-1fa0-4106-b744-d6666e5072fa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:44.807423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:44.808508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:44.809630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:44.810442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:44.812109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:44.813217 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:44.817048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:44.846314 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1216/2441] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:44.851346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:44.863728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534aabd-165b-4914-8efe-062ea5c5bc7e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:44.877886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:44.879446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:44.879973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:44.880702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:44.883190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:44.887798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:44.986293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1226/2442] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:44 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:44.988322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:45.017536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef89da4-d9c3-40ad-b0f0-09e7cf710648 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:45.145911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:45.146923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:45.147394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:45.147832 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:45.148840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:45.151574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:45.229690 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:45.231806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1217/2443] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:44 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:45.270167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a820214-a54a-4912-8c61-cb84940577a9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:45.298256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:45.299181 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:45.299475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:45.309845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:45.309845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:45.317133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:45.332966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1227/2444] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:45.335319 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:45.348329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede83f2-6cc9-4ad3-a68e-6c69bd2c712a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:45.354381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:45.355265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:45.355543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:45.355820 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:45.356732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:45.358491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d5c23-1ad2-46bd-9c75-aa91ec4ddd95 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:45.378335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1218/2445] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:45.383406 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:45.411165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667360f0-d939-4b5b-bfc9-717903599c4d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:45.426560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:45.428165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:45.428675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:45.429172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] PATH_INFO: `/v3/projects/bf1042b61955413fad23679484f9c891/users/e3d9262041784601bcb07c805038118c/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:45.481728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1042b61955413fad23679484f9c891, user_id=e3d9262041784601bcb07c805038118c, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:45.492285 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:45.542235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1228/2446] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:45 2022] PUT /identity/v3/projects/bf1042b61955413fad23679484f9c891/users/e3d9262041784601bcb07c805038118c/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:45.556700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:45.558839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:45.559214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:45.730323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] MFA Rules not processed for user `e3d9262041784601bcb07c805038118c`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:45.732385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:45.778236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:45.928767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1219/2447] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:45.941742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:45.945548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:45.986508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:46.058897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6f72-63ec-494b-8670-e226ce77132b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:46.063735 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:46.064758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:46.065421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:46.065796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:46.084089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:46.085724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:46.102499 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1229/2448] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:46.622609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:46.625477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:46.877143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e626e38-45aa-41b3-b750-42c8fee699ab None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:46.882777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:46.883758 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:46.884042 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:46.885173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] PATH_INFO: `/v3/users/bb3927ae30c54149974d3037c1022854` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:46.886765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3927ae30c54149974d3037c1022854)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:46.888849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:46.960102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3927ae30c54149974d3037c1022854'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:46.971593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3927ae30c54149974d3037c1022854'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:46.982161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3927ae30c54149974d3037c1022854'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:46.998970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3927ae30c54149974d3037c1022854'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:46.999838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-977d8f0e-2f58-4a20-bee6-023484e6c9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3927ae30c54149974d3037c1022854'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.008415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1220/2449] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:46 2022] DELETE /identity/v3/users/bb3927ae30c54149974d3037c1022854 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:47.310657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:47.328666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cb600-1218-4fb0-bac2-59aa2350ca85 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:47.434155 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:47.435743 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:47.436259 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:47.436759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] PATH_INFO: `/v3/projects/1b1fba96612b4ed0ae53b43cdd4e3528` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:47.439157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1fba96612b4ed0ae53b43cdd4e3528)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:47.442978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:47.522340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1fba96612b4ed0ae53b43cdd4e3528', 'request_id': 'req-31f03859-b3cf-4c57-b5c8-00d62f20a88f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.531244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1fba96612b4ed0ae53b43cdd4e3528', 'request_id': 'req-31f03859-b3cf-4c57-b5c8-00d62f20a88f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.539014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1fba96612b4ed0ae53b43cdd4e3528', 'request_id': 'req-31f03859-b3cf-4c57-b5c8-00d62f20a88f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.557279 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] The token cache is being invalidate because project 1b1fba96612b4ed0ae53b43cdd4e3528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:47.557602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.558315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-31f03859-b3cf-4c57-b5c8-00d62f20a88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:47.560014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1230/2450] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:47 2022] DELETE /identity/v3/projects/1b1fba96612b4ed0ae53b43cdd4e3528 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:47.565492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:47.565770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:47.566049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:47.599699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:47.602355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:47.658887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:47.741016 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1221/2451] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:47.749865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:47.750146 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:47.750418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:47.774189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:47.775696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:47.786173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1231/2452] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:47.792676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:47.813442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09dff-a3f3-4679-8607-d36251ab05fe None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:47.822333 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:47.825783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:47.825783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:47.825783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:47.825783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:47.825783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:47.828748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:47.862451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1222/2453] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:47.869606 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:47.914631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682dc69-fe4f-407f-b9a2-a74d8dd05b3f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:47.929793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:47.930996 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:47.931463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:47.931923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:47.933443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:47.937694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c2c2-2f23-45a9-8fe1-b0fc43e990f3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.016493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1232/2454] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:47 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:48.020008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:48.045938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f1792-28ce-4794-8b23-56d011ad1529 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:48.086790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.087254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.087674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.140922 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] MFA Rules not processed for user `e3d9262041784601bcb07c805038118c`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:48.142611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:48.156551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:48.189040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:48.190935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.191405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.192467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.193760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:48.198671 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.240962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1223/2455] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:48 2022] POST /identity/v3/users => generated 295 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:48.241299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1233/2456] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:48.243554 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:48.261634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894edd3-e8eb-4373-8053-5fb393e2ebfd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:48.272718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:48.273548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.274046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.274579 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.275920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:48.279393 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.304396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1224/2457] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:48.310908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:48.331043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad036ec7-5c7b-42da-b129-d2116a45cfce None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:48.339983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:48.341535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.342263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.343346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.345000 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:48.348126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461b577-c32f-46a9-9c2f-44e924542f32 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.368024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1234/2458] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:27:48.374299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:48.390191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8411b-2751-4879-9ebc-f50748bc4c9e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:48.397440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:48.398669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.399079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.399872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] PATH_INFO: `/v3/projects/084a82c286814b9793e16536a0ad88d7/users/8757fb1e0c604d6ba486cb213bf7e730/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.426125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084a82c286814b9793e16536a0ad88d7, user_id=8757fb1e0c604d6ba486cb213bf7e730, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:48.431595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.455640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1225/2459] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:48 2022] PUT /identity/v3/projects/084a82c286814b9793e16536a0ad88d7/users/8757fb1e0c604d6ba486cb213bf7e730/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:27:48.475551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.475982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.476520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.531992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] MFA Rules not processed for user `8757fb1e0c604d6ba486cb213bf7e730`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:48.534273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:48.683079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1235/2460] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:48.703382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:48.706732 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:48.834017 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19315-a5c8-48cb-b8c6-563178ead897 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:48.838659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:48.839374 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:48.839682 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:48.839960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:48.850862 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:48.853134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442ce27-8ca0-4732-a8de-0c550731d1d3 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:48.865918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1226/2461] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:51.877379 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:51.879194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:51.976058 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaea2d-dc8a-41c3-8d31-00ad03c49a3b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:51.984062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:51.985653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:51.986136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:51.986737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] PATH_INFO: `/v3/users/6205a3afef3c4f219afa9451ab5a24e3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:51.989553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6205a3afef3c4f219afa9451ab5a24e3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:51.993401 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:52.060581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:52.069797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6205a3afef3c4f219afa9451ab5a24e3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.086064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6205a3afef3c4f219afa9451ab5a24e3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.087004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6205a3afef3c4f219afa9451ab5a24e3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.101296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6205a3afef3c4f219afa9451ab5a24e3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.102066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6205a3afef3c4f219afa9451ab5a24e3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.128584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1236/2462] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:27:51 2022] DELETE /identity/v3/users/6205a3afef3c4f219afa9451ab5a24e3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:52.133935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:52.134733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:52.134733 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:52.157931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:52.159555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:52.169672 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:52.250738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1227/2463] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:52.262077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:52.263811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:52.336355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88666ad9-ed53-45fd-ac6e-152ccb853f3e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:52.340633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:52.341363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:52.341705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:52.342133 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:52.353014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:52.354550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4427-819b-4dc9-be27-0ceab4f3291c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:52.367100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1237/2464] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:52.807359 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:52.809921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:52.830178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8db500-346d-4587-9ba2-6d7adcaace04 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:52.837090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:52.838085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:52.838507 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:52.839061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] PATH_INFO: `/v3/projects/901cfd316e24473180a363e7974b2ed8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:52.841580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901cfd316e24473180a363e7974b2ed8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:52.843458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:52.915575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901cfd316e24473180a363e7974b2ed8', 'request_id': 'req-5cbe7414-469a-408c-8083-ef1a5a2ee457'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.924988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901cfd316e24473180a363e7974b2ed8', 'request_id': 'req-5cbe7414-469a-408c-8083-ef1a5a2ee457'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.936265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901cfd316e24473180a363e7974b2ed8', 'request_id': 'req-5cbe7414-469a-408c-8083-ef1a5a2ee457'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.955638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] The token cache is being invalidate because project 901cfd316e24473180a363e7974b2ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:27:52.955970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.956864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5cbe7414-469a-408c-8083-ef1a5a2ee457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:27:52.959828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1228/2465] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:27:52 2022] DELETE /identity/v3/projects/901cfd316e24473180a363e7974b2ed8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:52.966258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:52.967157 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:52.968055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:52.993998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:52.995375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:53.077315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1238/2466] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:27:53.188411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb393367-67d3-484d-8600-918c3d86423a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.189308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb393367-67d3-484d-8600-918c3d86423a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.189521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb393367-67d3-484d-8600-918c3d86423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.212009 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eb393367-67d3-484d-8600-918c3d86423a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:53.213604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb393367-67d3-484d-8600-918c3d86423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:53.222777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1229/2467] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:53.228188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.243291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789780e-c532-4ec1-a15f-cf09a0ec6999 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.248799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.249742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.250091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.250504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.251383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:27:53.251893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.253768 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.268679 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1239/2468] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:27:53.273791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb393367-67d3-484d-8600-918c3d86423a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.284594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb393367-67d3-484d-8600-918c3d86423a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.290013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.290715 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.291001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.291284 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.292039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.294069 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.333538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1230/2469] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:53 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:27:53.339047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.353514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca1526-6ac7-4c97-9247-0fa54c686734 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.429274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:53.454913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.455628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.455914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.456191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.456906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.458963 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.486326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1240/2470] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:53 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:53.491704 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.510194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb013a-3cb4-4374-96d6-b2ce5f2f24e4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.518106 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.519301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.519798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.520312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.521814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.525024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.549072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1231/2471] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:53.553728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.565207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b717-6ef0-4bd4-9300-cd8f7e22bfea admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.570841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.571615 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.571896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.572243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.573421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.575141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4814017-a022-4673-a6db-d8a75bc76870 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.589921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1241/2472] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:53.595019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.606669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93369e3-c77a-49c6-9330-dbfcb094d466 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:53.612356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:53.613276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.613601 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.614189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] PATH_INFO: `/v3/projects/c9786a5f12e5496abdb402949397bfcc/users/3313ba8045a74992bf40c0df23644527/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.633257 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9786a5f12e5496abdb402949397bfcc, user_id=3313ba8045a74992bf40c0df23644527, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:53.636145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:53.654340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1232/2473] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:53 2022] PUT /identity/v3/projects/c9786a5f12e5496abdb402949397bfcc/users/3313ba8045a74992bf40c0df23644527/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:53.665637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:53.665966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:53.666392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:53.701323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] MFA Rules not processed for user `3313ba8045a74992bf40c0df23644527`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:53.702640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:53.821061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1242/2474] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:53.828350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:53.922512 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:54.003903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907c1da-4146-4cd4-9d31-022aeec6d074 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:54.017647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:54.019348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.019838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.020928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.021906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:54.025406 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:54.144396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1233/2475] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:53 2022] POST /identity/v3/projects => generated 377 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:54.150012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:54.167707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afa89b-03a2-40d5-ba5b-fcc16d233593 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:54.364518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:54.365229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.366935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.367241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.368733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:54.372642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:54.420213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1243/2476] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:54 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:54.426993 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:54.459312 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647eeaa-1793-47de-b972-6f7755b0c67c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:54.495599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:54.499010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.500102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.502335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.506490 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:54.517436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:54.558799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1234/2477] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:54.560897 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:54.591528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a027-d2dd-47fe-a39a-3b21e2984c58 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:54.605166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:54.607625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.608329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.608975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.611867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:54.628744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ffbff-94ee-43ee-b157-5466111133f7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:54.695946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1244/2478] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:54.700014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:54.717205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790dbc-bd06-4614-a4cf-f362409857c2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:54.732049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:54.733988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.734407 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.734791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] PATH_INFO: `/v3/projects/7914b065542140b885e801c3866febb6/users/bf6085f60c19415098e4fcfb4015fbb1/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.775907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7914b065542140b885e801c3866febb6, user_id=bf6085f60c19415098e4fcfb4015fbb1, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:54.778943 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:54.806000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1235/2479] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:54 2022] PUT /identity/v3/projects/7914b065542140b885e801c3866febb6/users/bf6085f60c19415098e4fcfb4015fbb1/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:54.812160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:54.814624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:54.815839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:54.882520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] MFA Rules not processed for user `bf6085f60c19415098e4fcfb4015fbb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:54.884815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:54.950811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:55.057692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1245/2480] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:55.064271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.163041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063410fb-5963-495e-8255-e540c03c33ad admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.170872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.171566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.171843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.172160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.172891 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.174972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.210856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1236/2481] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:27:55 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:55.215439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.230194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377950f-5b4a-4e59-841e-e9f2a4f9f018 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.322992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.323711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.324002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.324345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.325070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.327632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.356932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1246/2482] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:27:55 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:27:55.358474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.371331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43be8c-3a59-402a-931f-e2fe41999397 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.376878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.377662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.377951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.378311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.379208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.380909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.395328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1237/2483] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:27:55.401831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.420421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a6c1-9e03-421f-a75e-6188b9232ced admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.428100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.429561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.430010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.430639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] PATH_INFO: `/v3/projects/483b19435bc84fb2ad95f07cf13ee25e/users/baad81081db24d8a8b28bc4d26cbd6e2/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.461114 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] RBAC: Authorizing `identity:create_grant(project_id=483b19435bc84fb2ad95f07cf13ee25e, user_id=baad81081db24d8a8b28bc4d26cbd6e2, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.466954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.494807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1247/2484] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:55 2022] PUT /identity/v3/projects/483b19435bc84fb2ad95f07cf13ee25e/users/baad81081db24d8a8b28bc4d26cbd6e2/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:55.499961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.520053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ee045-cf5c-4916-a191-adae78db3a24 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.590864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:55.600751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.601818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.602954 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.603364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.604964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.608306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9098-8478-43a6-872e-2bbc3fede6d7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.628478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1238/2485] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:27:55.635082 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.651728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c8977-c945-4fc8-8c5a-1b8a8420091e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:55.658050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:55.659883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.660168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.660446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] PATH_INFO: `/v3/projects/483b19435bc84fb2ad95f07cf13ee25e/users/baad81081db24d8a8b28bc4d26cbd6e2/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.663097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=483b19435bc84fb2ad95f07cf13ee25e, user_id=baad81081db24d8a8b28bc4d26cbd6e2, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:55.666043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:55.678995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1248/2486] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:27:55 2022] PUT /identity/v3/projects/483b19435bc84fb2ad95f07cf13ee25e/users/baad81081db24d8a8b28bc4d26cbd6e2/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:27:55.700092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:55.700635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:55.701014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:55.742591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] MFA Rules not processed for user `baad81081db24d8a8b28bc4d26cbd6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:27:55.744345 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:55.934486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1239/2487] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:27:55.950434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:55.960788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:27:56.091005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55f4b-c90a-4a62-aac6-c809a5705c7e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:56.095254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:56.096075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:56.096440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:56.096830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:56.145024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:27:56.182616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:56.183580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:56.196163 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1249/2488] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:27:59.207849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:27:59.218666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b8a4b-30e5-4b1d-a120-609467274b20 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:27:59.222913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:27:59.223632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:27:59.223956 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:27:59.224271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:27:59.304666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:27:59.306706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:27:59.318151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1240/2489] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:27:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:28:00.636753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:00.647913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c68dc-6e9b-4708-8c49-d29a123bd456 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:00.652640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:00.653324 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:00.653642 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:00.654961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:00.666717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:00.668365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4526d3-c7fd-44a2-b3fc-e7723fab0ca1 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:00.679998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1250/2490] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:28:06.307350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:06.512807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a470529-b1d0-40e7-bb38-232b621fc214 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:06.520441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:06.522109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:06.522593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:06.523890 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] PATH_INFO: `/v3/users/3313ba8045a74992bf40c0df23644527` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:06.525608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3313ba8045a74992bf40c0df23644527)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:06.529486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:06.563545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:06.604252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3313ba8045a74992bf40c0df23644527'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:06.617902 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3313ba8045a74992bf40c0df23644527'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:06.631366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3313ba8045a74992bf40c0df23644527'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:06.659641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3313ba8045a74992bf40c0df23644527'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:06.662014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3313ba8045a74992bf40c0df23644527'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:06.677696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1241/2491] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:06 2022] DELETE /identity/v3/users/3313ba8045a74992bf40c0df23644527 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:06.681840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:06.682804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:06.682804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:06.706728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:06.708077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:06.811787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1251/2492] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:28:06.825759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:06.828718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:06.974650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f115f0e-b1d1-49d6-8e49-cb42269e9ab1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:06.990900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:06.992452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:06.992725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:06.993030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:07.017011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:07.018942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:07.033841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1242/2493] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:07.526688 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:07.528666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:07.575312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4e42-73db-4cc1-bb54-ded7b81e9a28 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:07.598332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:07.602784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:07.603482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:07.605570 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] PATH_INFO: `/v3/projects/c9786a5f12e5496abdb402949397bfcc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:07.607686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9786a5f12e5496abdb402949397bfcc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:07.611182 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:07.692779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9786a5f12e5496abdb402949397bfcc', 'request_id': 'req-59f0faf1-5627-44e5-983b-7ef40450d3f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.708815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9786a5f12e5496abdb402949397bfcc', 'request_id': 'req-59f0faf1-5627-44e5-983b-7ef40450d3f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.722474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9786a5f12e5496abdb402949397bfcc', 'request_id': 'req-59f0faf1-5627-44e5-983b-7ef40450d3f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.754096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] The token cache is being invalidate because project c9786a5f12e5496abdb402949397bfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:07.754096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.754096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.754096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1252/2494] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:07 2022] DELETE /identity/v3/projects/c9786a5f12e5496abdb402949397bfcc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:07.754529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:07.771047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbb1a0-8e6e-4aa9-b2cf-5d48e3903c91 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:07.855915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:07.858372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:07.858949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:07.859417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] PATH_INFO: `/v3/users/bf6085f60c19415098e4fcfb4015fbb1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:07.861508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6085f60c19415098e4fcfb4015fbb1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:07.869023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:07.957534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:07.966226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6085f60c19415098e4fcfb4015fbb1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.979553 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6085f60c19415098e4fcfb4015fbb1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:07.997730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6085f60c19415098e4fcfb4015fbb1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.022342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6085f60c19415098e4fcfb4015fbb1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.023761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69839fa3-385b-44e0-9dca-315cd2c5a2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6085f60c19415098e4fcfb4015fbb1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.035868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1243/2495] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:07 2022] DELETE /identity/v3/users/bf6085f60c19415098e4fcfb4015fbb1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:28:08.286931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:08.289790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:08.331530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0faf1-5627-44e5-983b-7ef40450d3f4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:08.466798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:08.534764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:08.535696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:08.535972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:08.536264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] PATH_INFO: `/v3/users/8757fb1e0c604d6ba486cb213bf7e730` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:08.541889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8757fb1e0c604d6ba486cb213bf7e730)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:08.545717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:08.657675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8757fb1e0c604d6ba486cb213bf7e730'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.678889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8757fb1e0c604d6ba486cb213bf7e730'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.681024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8757fb1e0c604d6ba486cb213bf7e730'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.698172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8757fb1e0c604d6ba486cb213bf7e730'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.699426 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8757fb1e0c604d6ba486cb213bf7e730'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:08.745654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1253/2496] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:08 2022] DELETE /identity/v3/users/8757fb1e0c604d6ba486cb213bf7e730 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:28:08.751499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:08.752012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:08.752458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:08.788819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:08.793435 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:08.934501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1244/2497] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:08.944378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:08.949321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:08.972376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2402662-b6af-4c5d-9d27-4eb46c9e5190 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:08.981475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:08.983515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:08.984309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:08.984962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] PATH_INFO: `/v3/projects/7914b065542140b885e801c3866febb6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:08.987124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:08.987759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7914b065542140b885e801c3866febb6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:08.992168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:08.992759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:09.105470 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937811f-10dc-4a46-8aa7-cce38fbf847f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:09.117542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7914b065542140b885e801c3866febb6', 'request_id': 'req-f7288c93-6291-411b-95fc-5dc76f0058d2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.131583 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7914b065542140b885e801c3866febb6', 'request_id': 'req-f7288c93-6291-411b-95fc-5dc76f0058d2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.145278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7914b065542140b885e801c3866febb6', 'request_id': 'req-f7288c93-6291-411b-95fc-5dc76f0058d2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.175348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] The token cache is being invalidate because project 7914b065542140b885e801c3866febb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:09.176124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.177801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.182265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1254/2498] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:08 2022] DELETE /identity/v3/projects/7914b065542140b885e801c3866febb6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:09.188831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:09.192300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:09.239157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:09.241122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:09.242135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:09.243413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:09.248396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:09.325901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:09.330816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:09.375507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288c93-6291-411b-95fc-5dc76f0058d2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:09.385276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:09.387231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:09.388126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:09.388730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] PATH_INFO: `/v3/users/baad81081db24d8a8b28bc4d26cbd6e2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:09.391616 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] RBAC: Authorizing `identity:delete_user(user_id=baad81081db24d8a8b28bc4d26cbd6e2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:09.396873 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:09.442327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:09.445433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:09.487852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1245/2499] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:09.519030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baad81081db24d8a8b28bc4d26cbd6e2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.540343 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baad81081db24d8a8b28bc4d26cbd6e2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.541802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baad81081db24d8a8b28bc4d26cbd6e2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.558319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baad81081db24d8a8b28bc4d26cbd6e2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.559551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baad81081db24d8a8b28bc4d26cbd6e2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:09.610281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1255/2500] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:09 2022] DELETE /identity/v3/users/baad81081db24d8a8b28bc4d26cbd6e2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:10.197741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:10.202405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:10.230728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d01d-9496-4b9e-94b4-24212e9f0b18 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:10.274255 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:10.275822 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:10.399531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d29f41-6e80-4c74-8847-2c12cb0f627a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:10.402537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:10.404048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.404543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.405010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] PATH_INFO: `/v3/projects/084a82c286814b9793e16536a0ad88d7` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.407282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084a82c286814b9793e16536a0ad88d7)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:10.408081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:10.408993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.409420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.409693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] PATH_INFO: `/v3/projects/483b19435bc84fb2ad95f07cf13ee25e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.411320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:10.411746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483b19435bc84fb2ad95f07cf13ee25e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:10.413918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:10.510388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084a82c286814b9793e16536a0ad88d7', 'request_id': 'req-472fa07e-9bb4-4340-9841-2646e409aa81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.521543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483b19435bc84fb2ad95f07cf13ee25e', 'request_id': 'req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.522171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084a82c286814b9793e16536a0ad88d7', 'request_id': 'req-472fa07e-9bb4-4340-9841-2646e409aa81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.529232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483b19435bc84fb2ad95f07cf13ee25e', 'request_id': 'req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.537131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084a82c286814b9793e16536a0ad88d7', 'request_id': 'req-472fa07e-9bb4-4340-9841-2646e409aa81'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.538648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483b19435bc84fb2ad95f07cf13ee25e', 'request_id': 'req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.543681 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:10.562247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] The token cache is being invalidate because project 483b19435bc84fb2ad95f07cf13ee25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:10.563130 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] The token cache is being invalidate because project 084a82c286814b9793e16536a0ad88d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:10.563396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.563572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.564486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-472fa07e-9bb4-4340-9841-2646e409aa81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.565136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-91808c1c-c4f0-4c75-bdcb-5cac7b967ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:10.575533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1256/2502] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:10 2022] DELETE /identity/v3/projects/483b19435bc84fb2ad95f07cf13ee25e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:28:10.576010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1246/2502] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:10 2022] DELETE /identity/v3/projects/084a82c286814b9793e16536a0ad88d7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:10.579258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.579770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.580263 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.615095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:10.616430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:10.726493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1257/2503] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:10.737762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.738576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.739804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.794306 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:10.797572 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:10.836960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1247/2504] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:10.837256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:10.878068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a130-1b80-434d-b9c3-9b378e81e981 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:10.894303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:10.895732 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.896249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.896713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.898164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:28:10.898751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:10.901811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:10.931272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1258/2505] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:28:10.932884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:10.966703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8325c-3271-4d69-bb83-694567b94893 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:10.990817 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:10.991868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:10.992400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:10.992915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:10.994354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:10.998318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.075784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1248/2506] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:28:10 2022] POST /identity/v3/projects => generated 399 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:28:11.081317 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:11.101559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c3d9-a1aa-44b1-9865-3d4c7643c185 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:11.190153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:11.191339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.191912 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.192489 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.193827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:11.197631 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.238267 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1259/2507] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:28:11 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:28:11.239806 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:11.252547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e46b3d-dc57-40e9-a0fb-8aa1f7bb9733 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:11.258440 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:11.259386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.259722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.260263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.261045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:11.263125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.278355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1249/2508] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:11.284307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:11.295938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d1cc-5bf3-48e8-8154-2d6ec1857638 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:11.301537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:11.302782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.303233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.303403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.304081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:11.306247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9ba18-1479-446f-88fb-d0eb08df06f0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.323910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1260/2509] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:11.328577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:11.340043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25fa29-935b-4553-8479-4775b772a961 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:11.345696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:11.346655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.346950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.347227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] PATH_INFO: `/v3/projects/986834ce5df540f194c3f27863017b31/users/7b384650a3ca48a8b9861f8af33b5fa9/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.366059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:11.370676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=986834ce5df540f194c3f27863017b31, user_id=7b384650a3ca48a8b9861f8af33b5fa9, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:11.373600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.396740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1250/2510] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:28:11 2022] PUT /identity/v3/projects/986834ce5df540f194c3f27863017b31/users/7b384650a3ca48a8b9861f8af33b5fa9/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:28:11.416691 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.417025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.417478 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.455471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] MFA Rules not processed for user `7b384650a3ca48a8b9861f8af33b5fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:11.457378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:11.538599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:11.565165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1261/2511] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:11.581858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:11.584133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:11.694413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba32f5-f37b-4228-8605-e0f8f02e8fdd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:11.699607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:11.700254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:11.700599 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:11.700972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:11.715282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:11.719099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3e9d8-3d24-4dd7-a237-0b5c4e93d9ce service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:11.739482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1251/2512] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:20.100301 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:20.102712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:20.232964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55b7e2-05af-4d2e-addc-3e500bfec295 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:20.246635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:20.247559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:20.248669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:20.249939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:20.260805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:20.344135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:20.344791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:20.345152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:20.373898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:20.375219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:20.380779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:20.384115 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:20.406268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1262/2513] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:20.464892 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1252/2514] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:28:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:28:20.510275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:20.514034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:20.642143 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:20.662917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f91e-0d59-4b8a-9fde-187cd96ebb6d service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:20.678628 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:20.679955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:20.680564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:20.681106 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:20.703238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:20.703238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28385f-9b7b-4342-be98-8cd1359678bf service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:20.716653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1263/2515] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:28:20.789905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841ff63-e051-4c1d-83cd-6704ee1914e0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:20.798418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:20.800256 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:20.800866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:20.801493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] PATH_INFO: `/v3/users/7b384650a3ca48a8b9861f8af33b5fa9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:20.804050 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b384650a3ca48a8b9861f8af33b5fa9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:20.808433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:20.882552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:20.891790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b384650a3ca48a8b9861f8af33b5fa9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:20.904603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b384650a3ca48a8b9861f8af33b5fa9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:20.916450 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b384650a3ca48a8b9861f8af33b5fa9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:20.938946 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b384650a3ca48a8b9861f8af33b5fa9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:20.940521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b384650a3ca48a8b9861f8af33b5fa9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:20.951346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1253/2516] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:20 2022] DELETE /identity/v3/users/7b384650a3ca48a8b9861f8af33b5fa9 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:20.956924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488051-eaec-48c6-b7ce-c6413a5bc0ba None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:20.957211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488051-eaec-48c6-b7ce-c6413a5bc0ba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:20.957985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488051-eaec-48c6-b7ce-c6413a5bc0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:20.983344 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-10488051-eaec-48c6-b7ce-c6413a5bc0ba None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:20.986577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-10488051-eaec-48c6-b7ce-c6413a5bc0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:21.031271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:21.114169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1264/2517] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:21.123616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:21.124426 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:21.329719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c77ffa-dc95-4db8-b832-8180dbf9b1d9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:21.345152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:21.346180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:21.346318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:21.346660 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:21.364769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:21.366472 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:21.378743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1254/2518] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:28:21.783037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:21.783419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:21.784232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:21.831564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:21.839674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:22.063088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1265/2519] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:28:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:22.096536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:22.099420 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:22.220559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba5b86-9b84-446e-9d15-afd4d1294ec4 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:22.227007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:22.227865 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:22.228206 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:22.228485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:22.239778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:22.241684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2fce1-41e2-4af4-95d6-7f9ac86f4032 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:22.252696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1255/2520] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:28:22.466371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:22.467804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:22.487731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceebba8-efe7-4e20-916f-403918f8fdc1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:22.499600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:22.501472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:22.502051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:22.503063 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] PATH_INFO: `/v3/projects/986834ce5df540f194c3f27863017b31` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:22.505903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986834ce5df540f194c3f27863017b31)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:22.510645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:22.611790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986834ce5df540f194c3f27863017b31', 'request_id': 'req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:22.622933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9d5d-ec4c-4019-957b-d775022d06e2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:22.623754 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986834ce5df540f194c3f27863017b31', 'request_id': 'req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:22.624562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9d5d-ec4c-4019-957b-d775022d06e2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:22.625079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a9d5d-ec4c-4019-957b-d775022d06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:22.635355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986834ce5df540f194c3f27863017b31', 'request_id': 'req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:22.652436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-572a9d5d-ec4c-4019-957b-d775022d06e2 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:22.655131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] The token cache is being invalidate because project 986834ce5df540f194c3f27863017b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:22.655557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:22.655965 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-572a9d5d-ec4c-4019-957b-d775022d06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:22.656325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:22.658757 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1266/2521] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:22 2022] DELETE /identity/v3/projects/986834ce5df540f194c3f27863017b31 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:22.764077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:22.796236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1256/2522] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:22.821360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:22.823112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:22.955213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:23.012310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2901f5-35f1-49cd-a0bc-5292e4ab9388 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:23.025209 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:23.026485 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:23.027196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:23.027737 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:23.045904 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:23.049066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:23.067812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1267/2523] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 16:28:23.558552 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:23.558845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:23.559128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:23.581603 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:23.582915 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:23.590866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1257/2524] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:28:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:23.606724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:23.621325 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a960-a98b-4738-a02a-61e55baa2a85 service swift] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:23.629470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:23.629470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:23.629957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:23.629957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:23.641006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:23.643031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d36cce-421c-4f82-95da-9d678462119d service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:23.654292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1268/2525] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:28:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:28:26.767940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:26.769878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:26.918485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98d9e-91ba-44a7-b005-65d75c9451cf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:26.928742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:26.930385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:26.930847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:26.931324 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] PATH_INFO: `/v3/users/e3d9262041784601bcb07c805038118c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:26.934428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d9262041784601bcb07c805038118c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:26.938375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:27.021235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d9262041784601bcb07c805038118c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.032647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d9262041784601bcb07c805038118c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.044638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d9262041784601bcb07c805038118c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.070794 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d9262041784601bcb07c805038118c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.071568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d9262041784601bcb07c805038118c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.081611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1258/2526] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:28:26 2022] DELETE /identity/v3/users/e3d9262041784601bcb07c805038118c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:27.088856 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:27.089497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:27.089801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:27.140247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:27.143218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:27.309651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1269/2527] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:28:27.340199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:27.343451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:27.468037 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9920fa-6695-4fd6-9890-afe9542aac9d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:27.489215 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:27.495776 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:27.497980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:27.498578 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:27.516449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:27.518404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a0b7b-d3f8-4a68-bc04-8c84f4747b56 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:27.536938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1259/2528] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:27.780153 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:27.782712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:27.804956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9417bf-9dd8-4df4-aa9d-664e664cc4a6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:27.816194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:27.817803 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:27.818420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:27.819139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] PATH_INFO: `/v3/projects/bf1042b61955413fad23679484f9c891` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:27.821549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1042b61955413fad23679484f9c891)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:27.825557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:27.887298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1042b61955413fad23679484f9c891', 'request_id': 'req-c870914f-e35b-4774-8610-72dd18da1d5e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.895546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1042b61955413fad23679484f9c891', 'request_id': 'req-c870914f-e35b-4774-8610-72dd18da1d5e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.903969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1042b61955413fad23679484f9c891', 'request_id': 'req-c870914f-e35b-4774-8610-72dd18da1d5e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.925393 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] The token cache is being invalidate because project bf1042b61955413fad23679484f9c891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:28:27.925826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.926701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c870914f-e35b-4774-8610-72dd18da1d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:28:27.928614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1270/2529] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:28:27 2022] DELETE /identity/v3/projects/bf1042b61955413fad23679484f9c891 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:27.934767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:27.935905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:27.935905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:27.958762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:27.959996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:27.968186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:28.060455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1260/2530] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:28.070477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359a36-7604-4043-ab43-258ba71a114a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.070806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359a36-7604-4043-ab43-258ba71a114a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.071177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359a36-7604-4043-ab43-258ba71a114a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.096112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-99359a36-7604-4043-ab43-258ba71a114a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:28.097536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-99359a36-7604-4043-ab43-258ba71a114a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:28.108271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1271/2531] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:28.115136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.127182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1bed-9a51-462f-a516-3ca678943c68 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.133756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.134765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.135159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.135536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.136518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:28:28.136936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.138955 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.154540 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1261/2532] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:28:28.159677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99359a36-7604-4043-ab43-258ba71a114a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.170916 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99359a36-7604-4043-ab43-258ba71a114a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.176531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.177456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.177850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.178320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.179160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.181329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.224872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1272/2533] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:28:28 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:28:28.231499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.245589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd975d-837b-412b-b860-2b953d7f7847 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.336722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.337981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.338527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.339096 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.340668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.344600 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.378363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1262/2534] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:28:28 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:28:28.384774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.396731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092f088-222d-4cf2-9389-46012da3d5ba admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.403025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.403858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.404177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.404530 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.405376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.407557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.420012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1273/2535] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:28.428174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.447108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4804d3-81b1-4234-92b0-a5521f4b8998 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.453804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.454797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.455271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.455614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.458054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.460893 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dbfec-f47e-4064-be17-60153096df84 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.481302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1263/2536] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:28.486176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.506952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09a1-4fff-468d-b2ab-a541ac3c2b88 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.509638 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.511804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.512319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.512833 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] PATH_INFO: `/v3/projects/90317adf2c86461d96439dc48441b6c8/users/6a52a93774cd48868f9b5656821b0f12/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.540172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=90317adf2c86461d96439dc48441b6c8, user_id=6a52a93774cd48868f9b5656821b0f12, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.543702 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.546668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:28.566224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1274/2537] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:28:28 2022] PUT /identity/v3/projects/90317adf2c86461d96439dc48441b6c8/users/6a52a93774cd48868f9b5656821b0f12/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:28:28.573166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ed5ca-fdd7-4c62-a7fd-e8c8e81fe35e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.574024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ed5ca-fdd7-4c62-a7fd-e8c8e81fe35e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.574486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ed5ca-fdd7-4c62-a7fd-e8c8e81fe35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.603498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d88ed5ca-fdd7-4c62-a7fd-e8c8e81fe35e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:28.605242 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d88ed5ca-fdd7-4c62-a7fd-e8c8e81fe35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:28.759436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1264/2538] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:28:28.777578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:28.779858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:28.882634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874c9f6-40ca-42d7-bd11-d761697d4c9c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:28.889197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:28.890437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:28.890926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:28.891408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:28.915262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:28.918494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:28.935552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1275/2539] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:28:36.182190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.183156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.183156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.229039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] MFA Rules not processed for user `6a52a93774cd48868f9b5656821b0f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:36.230863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:36.240224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:36.337850 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1265/2540] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:36.342762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:36.354747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bd4e5-80df-43c2-9753-4628c48d66d1 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:36.372308 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:36.373416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.373651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.373985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.374817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:36.377796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:36.454315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1276/2541] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:28:36 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:28:36.461125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:36.484164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b682bc-73d8-4bb3-8f28-c4e5c56e981d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:36.618724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:36.620098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.620708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.621208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.622884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:36.626928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:36.680126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1266/2542] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:28:36 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:28:36.685770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:36.727429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46b36-dd3a-4272-99ae-a2089c76028b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:36.737594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:36.739140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.739610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.740962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.744979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:36.748504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:36.772973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1277/2543] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:36.776058 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:36.791461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85a623-b0a0-48a3-ba52-cd20ce5f691b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:36.797559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:36.798668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.799054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.799343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] PATH_INFO: `/v3/projects/ad3e87f106944a95990be3ec4e4da21f/users/069913534b5c4d5c9c96acae8f24f84b/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.819886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3e87f106944a95990be3ec4e4da21f, user_id=069913534b5c4d5c9c96acae8f24f84b, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:36.823177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:36.842988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1267/2544] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:28:36 2022] PUT /identity/v3/projects/ad3e87f106944a95990be3ec4e4da21f/users/069913534b5c4d5c9c96acae8f24f84b/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:36.849777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:36.870088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a22c8-1fe6-4b71-806b-334c17176ff8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:36.985352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:36.986216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:36.986503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:36.986779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:36.987533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:36.989761 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc525d-9a8d-4ee7-812d-930a41205b74 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:37.017397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1278/2545] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:28:37.021026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:37.034508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8b7d-3cfd-4c35-b9d6-ca54438891ed admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:37.040674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:37.041869 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:37.042155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:37.042438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] PATH_INFO: `/v3/projects/ad3e87f106944a95990be3ec4e4da21f/users/069913534b5c4d5c9c96acae8f24f84b/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:37.047799 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3e87f106944a95990be3ec4e4da21f, user_id=069913534b5c4d5c9c96acae8f24f84b, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:37.055034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:37.074706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1268/2546] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:28:37 2022] PUT /identity/v3/projects/ad3e87f106944a95990be3ec4e4da21f/users/069913534b5c4d5c9c96acae8f24f84b/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:28:44.692966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1932f-f6a9-45ea-a527-831158421a2e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:44.693766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1932f-f6a9-45ea-a527-831158421a2e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:44.694125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1932f-f6a9-45ea-a527-831158421a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:44.731247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aee1932f-f6a9-45ea-a527-831158421a2e None None] MFA Rules not processed for user `069913534b5c4d5c9c96acae8f24f84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:44.733156 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aee1932f-f6a9-45ea-a527-831158421a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:44.800867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:44.844071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1279/2547] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:44.860363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:28:44.863937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:44.976201 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:28:44.999303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df6b2-fe06-4e56-add0-3103373bafab admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:28:45.007539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:28:45.008538 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:45.008928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:45.009328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:45.092094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:28:45.094292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fee54-98a9-452b-82d2-0e93dc45905b service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:28:45.130173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1269/2548] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:28:46.816712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c1b3-3d42-40ea-8118-1b9d507f34f3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:46.817443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c1b3-3d42-40ea-8118-1b9d507f34f3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:46.817443 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c1b3-3d42-40ea-8118-1b9d507f34f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:46.839990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e087c1b3-3d42-40ea-8118-1b9d507f34f3 None None] MFA Rules not processed for user `6a52a93774cd48868f9b5656821b0f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:46.843635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e087c1b3-3d42-40ea-8118-1b9d507f34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:46.853464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1280/2549] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:53.300905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:28:53.301795 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:28:53.302532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:28:53.343520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] MFA Rules not processed for user `2d1eb89923784f1c83d2077c6b6926e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:28:53.345834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:28:53.453132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1270/2550] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:28:56.832885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=10 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:03.441187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=54 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:11.369639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:11.370566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:11.370566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:11.402410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] MFA Rules not processed for user `c7e1898925cd4feea900627bc0ee8a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:29:11.404247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:11.511527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1281/2551] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:29:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:29:21.498220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=25 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:23.352127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:23.354995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:23.472021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7adef-9350-42ba-a188-6be2ad3db6aa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:23.481185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:23.482081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:23.482462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:23.482841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:23.485777 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:23.502202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:23.503604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:23.516771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1271/2552] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:29:26.002994 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:26.021180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e174-1685-4115-bffe-19a47a076b81 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:26.025292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:26.026006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:26.026299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:26.026723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:26.029086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:26.047198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:26.048729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8d9b1-b503-4f48-9b8d-c637e084733e service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:26.064100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1282/2553] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:29:33.520146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=26 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:33.538842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:33.552227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358db0d-ecbc-4507-88e0-5917da3cf462 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:33.557398 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:33.558375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:33.558663 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:33.559199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:33.573770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:33.576769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:33.597395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1272/2554] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:29:36.056147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:36.847750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:36.848273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:36.848769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:36.882855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] MFA Rules not processed for user `644c0a532fa747c7abf4d53759e89181`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:29:36.884135 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:36.985405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1283/2555] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:29:38.516184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:38.520089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:38.647093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a4fe-92da-4fb3-a341-82a6bb40921d service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:38.656129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:38.660645 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:38.661041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:38.661493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:38.664250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:38.679086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:38.680931 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:38.692980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1273/2556] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:29:39.621194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:39.623171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:39.645090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778fcb9-e295-4764-995f-c587fe5a58f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:39.652654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:39.653392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:39.653682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:39.653958 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:39.655944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:39.669931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:39.671936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:39.683906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1284/2557] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:29:43.091052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:43.101171 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898e18-2148-4c50-876f-a19d8810d03a service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:43.107809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:43.108930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:43.109574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:43.109863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:43.112563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:43.128205 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:43.129796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:43.142655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1274/2558] 10.222.0.66 () {60 vars in 1396 bytes} [Mon Jul 25 16:29:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:29:49.678685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=31 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:29:50.210418 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:29:50.228352 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ae14f-d8e9-4bb5-bc6e-eb6587355356 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:29:50.234011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:29:50.234569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:29:50.235094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:29:50.235428 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:29:50.237414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:29:50.252888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:29:50.254949 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:29:50.269788 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1285/2559] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:29:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:29:53.136680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=44 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:00.265030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:03.021853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:03.043453 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650576f1-effb-4fe4-95dc-5840f9bcab0c service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:03.053984 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:03.055095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:03.055653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:03.056261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:03.060200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:03.086607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:03.089200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:03.107651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1275/2560] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:11.274589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:11.305514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28181ff-9ede-4631-b4b6-93fa9e2d9eb0 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:11.317095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:11.318381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:11.318925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:11.319463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:11.323210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:11.350033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:11.353623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68b651-bc39-4381-8ea3-4ed71e4d8fda service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:11.380456 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1286/2561] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:12.137369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:12.148989 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967644-333a-4da3-b4c0-c4e05a698ac4 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:12.154963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:12.156838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:12.157465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:12.157913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:12.161551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:12.298923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:12.301743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:12.337005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1276/2562] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:30:16.843670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:16.844398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:16.844736 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:16.883804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:16.886614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:17.004319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1287/2563] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:17.075018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:17.077192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:17.215900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f003d-26f3-44e4-837c-efdaec1d98ec service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:17.223493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:17.224792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:17.225482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:17.226065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:17.242683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:17.245926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:17.263567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1277/2564] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:30:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:30:18.794906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:18.798593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:18.826781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df56d-2564-408c-9419-7b705e6ef37e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:18.833780 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:18.834806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:18.835148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:18.835488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] PATH_INFO: `/v3/users/2d1eb89923784f1c83d2077c6b6926e9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:18.837410 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d1eb89923784f1c83d2077c6b6926e9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:18.839766 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:18.915044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d1eb89923784f1c83d2077c6b6926e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:18.930225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d1eb89923784f1c83d2077c6b6926e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:18.931439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d1eb89923784f1c83d2077c6b6926e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:18.944085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d1eb89923784f1c83d2077c6b6926e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:18.945028 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-256483bf-22ba-458a-99b3-2520a678050e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d1eb89923784f1c83d2077c6b6926e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:18.969886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1288/2565] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:30:18 2022] DELETE /identity/v3/users/2d1eb89923784f1c83d2077c6b6926e9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:19.335208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:19.477898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590bdf2-af1b-4192-821b-496c4cd02fd5 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:19.555746 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:19.627998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:19.629276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:19.629748 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:19.630181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] PATH_INFO: `/v3/projects/18b6e0eaf4eb48868a683c7769d3df7a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:19.632173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b6e0eaf4eb48868a683c7769d3df7a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:19.636079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:19.731614 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b6e0eaf4eb48868a683c7769d3df7a', 'request_id': 'req-7957dac9-2863-43a1-b49b-8110754a24af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:19.743105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b6e0eaf4eb48868a683c7769d3df7a', 'request_id': 'req-7957dac9-2863-43a1-b49b-8110754a24af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:19.754678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b6e0eaf4eb48868a683c7769d3df7a', 'request_id': 'req-7957dac9-2863-43a1-b49b-8110754a24af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:19.823950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] The token cache is being invalidate because project 18b6e0eaf4eb48868a683c7769d3df7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:30:19.824910 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:19.826774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7957dac9-2863-43a1-b49b-8110754a24af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:19.830431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1278/2566] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:30:19 2022] DELETE /identity/v3/projects/18b6e0eaf4eb48868a683c7769d3df7a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:30:19.837925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:19.838524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:19.839051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.011433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:20.013584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:20.045858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:20.127794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1289/2567] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:30:20.136462 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.136918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.137277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.172076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:20.173895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:20.189395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1279/2568] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:20.193140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.205577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cb4b5-66b5-47ff-ac33-59b546ad68ee None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.212103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.212915 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.213260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.213607 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.214475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:30:20.214892 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.217020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.236117 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1290/2569] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:30:20.242392 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.257604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b84c9-1c75-4c2c-9af4-a56c6454bb10 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.263384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.264129 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.264416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.264718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.265527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.267610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.319504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1280/2570] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:30:20 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:30:20.324258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.345785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb62ce-b758-4455-9700-aefdce70c067 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.461140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.462237 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.462413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.462701 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.463624 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.466031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.513823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1291/2571] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:30:20 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:30:20.518698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.539317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286224c-d183-4323-90f6-b4f873618e74 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.549260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.550544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.551246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.552048 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.553542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.557317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.579290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1281/2572] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:30:20.585651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.599408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a905a25-51ad-477c-941c-dbc6d1213c8c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.605953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.606662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.606948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.607225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.608152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.610047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511dbe5-62cd-4bb0-a17b-7538ae218eed admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.632214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1292/2573] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:30:20.634544 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.656227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246207e5-03f6-4a0c-bcd3-e6e8343d358a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:20.665583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:20.666524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.666809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.667232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] PATH_INFO: `/v3/projects/4501489e56744cf98e49895b6d5aa6f8/users/37e7fecdaae94842b6917152fb7f1598/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.696905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4501489e56744cf98e49895b6d5aa6f8, user_id=37e7fecdaae94842b6917152fb7f1598, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:20.696905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:20.725679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1282/2574] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:30:20 2022] PUT /identity/v3/projects/4501489e56744cf98e49895b6d5aa6f8/users/37e7fecdaae94842b6917152fb7f1598/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:20.735022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:20.735022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:20.735496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:20.781378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:20.783634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:20.955515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1293/2575] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:20.973131 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:20.978452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:21.083234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:21.174394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234c6f-8045-4a51-ad27-06bb65a3e533 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:21.182711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:21.184105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:21.185281 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:21.185848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:21.207827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:21.211411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:21.235303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1283/2576] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:23.262985 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:23.265654 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:23.299459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548727d-c4df-4fa6-9685-cc812d24e1d3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:23.308260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:23.310346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:23.311180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:23.311872 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] PATH_INFO: `/v3/users/c7e1898925cd4feea900627bc0ee8a7a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:23.315875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e1898925cd4feea900627bc0ee8a7a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:23.320110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:23.563215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:23.612198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e1898925cd4feea900627bc0ee8a7a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:23.632398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e1898925cd4feea900627bc0ee8a7a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:23.633535 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e1898925cd4feea900627bc0ee8a7a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:23.646828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e1898925cd4feea900627bc0ee8a7a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:23.648329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6422474a-9ddf-49e7-98ba-d1de47d7e2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e1898925cd4feea900627bc0ee8a7a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:23.682896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1294/2577] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:30:23 2022] DELETE /identity/v3/users/c7e1898925cd4feea900627bc0ee8a7a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:24.024251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:24.069125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01c32e-aedf-4f88-8248-ab9bbf4db7fe service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:24.230209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:24.232018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:24.232504 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:24.232900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] PATH_INFO: `/v3/projects/bd9b5f7127564335a847ab40761124df` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:24.234729 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9b5f7127564335a847ab40761124df)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:24.237311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:24.368783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9b5f7127564335a847ab40761124df', 'request_id': 'req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:24.395422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9b5f7127564335a847ab40761124df', 'request_id': 'req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:24.411357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9b5f7127564335a847ab40761124df', 'request_id': 'req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:24.441448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] The token cache is being invalidate because project bd9b5f7127564335a847ab40761124df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:30:24.441448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:24.441448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-200e565b-ecd1-4d57-9fda-6e8802aeb2e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:24.442218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1284/2578] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:30:24 2022] DELETE /identity/v3/projects/bd9b5f7127564335a847ab40761124df => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:24.461713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:24.463214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:24.466302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:24.525677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:24.527017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:24.691310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1295/2579] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:24.693924 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:24.694764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:24.695315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:24.756767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:24.759173 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:24.776999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1285/2580] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:24.784207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:24.802924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb71f42-89e3-449d-93f4-ed726b7c4e80 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:24.812504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:24.814047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:24.814495 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:24.815187 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:24.816454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:30:24.817910 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:24.819857 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:24.845477 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1296/2581] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:30:24.851372 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:24.876498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac31e7-95cf-470b-a9c7-7afb5d8ac68a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:24.886023 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:24.887772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:24.888062 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:24.888699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:24.890197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:24.893609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:24.967155 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1286/2582] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:30:24 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:30:24.972474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:25.003315 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29979de-6863-41c2-a95c-4752056a6125 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:25.106421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:25.109029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.109327 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.109695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.111084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:25.114408 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:25.150022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1297/2583] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:30:24 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:30:25.156322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:25.186068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba77c4-0836-4076-934a-795f4a6d3f93 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:25.211075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:25.218170 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.219438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.220221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.222011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:25.225563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:25.256961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1287/2584] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:30:25.261461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:25.274772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26494-754d-4fba-8d72-9e8877cb0e97 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:25.280657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:25.281641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.282178 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.282556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.283454 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:25.285894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937338b-543b-47c4-913f-0dd4493b7e7a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:25.307217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1298/2585] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:30:25.311639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:25.332329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714381e-417b-4e36-be0a-54b7e377e4e3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:25.340072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:25.341520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.342007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.342397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] PATH_INFO: `/v3/projects/ee2c762eec4646afbd0ea4d71a58894d/users/d3eebc82ce654013998f58585c123a0e/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.361828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:25.365899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2c762eec4646afbd0ea4d71a58894d, user_id=d3eebc82ce654013998f58585c123a0e, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:25.371597 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:25.397417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1288/2586] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:30:25 2022] PUT /identity/v3/projects/ee2c762eec4646afbd0ea4d71a58894d/users/d3eebc82ce654013998f58585c123a0e/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:25.401131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.401525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.401813 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.431770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:25.437700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:25.444118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:25.563618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1299/2587] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:25.575938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:25.579367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:25.719903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675f0f-4c6a-473f-bc6c-e0f454dcb337 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:25.728231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:25.730332 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:25.731033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:25.732388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:25.756395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:25.762365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc540081-8429-47bc-8d40-87ac26d352b6 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:25.791567 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1289/2588] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:31.932955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:31.937371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:31.984806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648cc09-d3c8-4615-9b76-e7d7ec4f5761 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:32.010186 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:32.012093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:32.012578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:32.019596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] PATH_INFO: `/v3/users/55d7bbda47b64493a840fef8c6c40431` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:32.022609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d7bbda47b64493a840fef8c6c40431)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:32.031107 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:32.146540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d7bbda47b64493a840fef8c6c40431'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:32.173435 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d7bbda47b64493a840fef8c6c40431'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:32.174661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d7bbda47b64493a840fef8c6c40431'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:32.190697 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d7bbda47b64493a840fef8c6c40431'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:32.191995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d7bbda47b64493a840fef8c6c40431'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:32.265474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1300/2589] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:30:31 2022] DELETE /identity/v3/users/55d7bbda47b64493a840fef8c6c40431 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:32.292385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:32.292385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:32.293424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:32.342881 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] MFA Rules not processed for user `37e7fecdaae94842b6917152fb7f1598`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:32.347824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:32.564636 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1290/2590] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:30:32.596139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:32.601213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:32.777389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:32.780193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:32.961514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31db-33e1-4142-836e-894438db5ec7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:32.974988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:32.976502 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:32.977447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:32.978073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:32.990793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:32.992979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:33.010972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1301/2591] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:33.115831 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e7608-532a-4ae0-b27a-09562a0cf6ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:33.140348 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:33.141757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:33.143901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:33.144727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] PATH_INFO: `/v3/projects/966cf8bbbc4540a596f615498331465a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:33.147300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966cf8bbbc4540a596f615498331465a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:33.153964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:33.172126 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:33.390568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966cf8bbbc4540a596f615498331465a', 'request_id': 'req-5d984f09-7785-48c3-8c51-a8c62e92e719'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.420148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966cf8bbbc4540a596f615498331465a', 'request_id': 'req-5d984f09-7785-48c3-8c51-a8c62e92e719'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.442142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966cf8bbbc4540a596f615498331465a', 'request_id': 'req-5d984f09-7785-48c3-8c51-a8c62e92e719'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.486473 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] The token cache is being invalidate because project 966cf8bbbc4540a596f615498331465a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:30:33.486870 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.488098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.500684 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:33.505202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1291/2592] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:30:32 2022] DELETE /identity/v3/projects/966cf8bbbc4540a596f615498331465a => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:33.524507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1eb51-0660-4ad4-9898-a6623ef8be99 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:33.530462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:33.639177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:33.641722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:33.642201 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:33.642671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] PATH_INFO: `/v3/users/16b7350f73e14634b6dfa6ad3a72d45c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:33.652086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b7350f73e14634b6dfa6ad3a72d45c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:33.654531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:33.770984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b7350f73e14634b6dfa6ad3a72d45c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.785806 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b7350f73e14634b6dfa6ad3a72d45c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.786923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b7350f73e14634b6dfa6ad3a72d45c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.801050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b7350f73e14634b6dfa6ad3a72d45c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.802836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7a72085a-1fe1-4b07-90ce-c9afad4d8235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b7350f73e14634b6dfa6ad3a72d45c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:33.905961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1302/2593] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:30:33 2022] DELETE /identity/v3/users/16b7350f73e14634b6dfa6ad3a72d45c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:35.072675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:35.074907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:35.160788 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984f09-7785-48c3-8c51-a8c62e92e719 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:35.334353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:35.336802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:35.339189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:35.340046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] PATH_INFO: `/v3/projects/9ce8951f40384a89815320ea0108e5e8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:35.343525 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce8951f40384a89815320ea0108e5e8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:35.348109 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:35.522933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce8951f40384a89815320ea0108e5e8', 'request_id': 'req-b62f322b-98d0-405d-a295-97bcb36d729b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:35.552139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce8951f40384a89815320ea0108e5e8', 'request_id': 'req-b62f322b-98d0-405d-a295-97bcb36d729b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:35.574689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce8951f40384a89815320ea0108e5e8', 'request_id': 'req-b62f322b-98d0-405d-a295-97bcb36d729b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:35.635482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] The token cache is being invalidate because project 9ce8951f40384a89815320ea0108e5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:30:35.635772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:35.638858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:30:35.645074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1292/2594] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:30:35 2022] DELETE /identity/v3/projects/9ce8951f40384a89815320ea0108e5e8 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:37.360031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4263c17-b93c-4b84-ab95-c8a4cf6129d0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:37.360031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4263c17-b93c-4b84-ab95-c8a4cf6129d0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:37.360031 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4263c17-b93c-4b84-ab95-c8a4cf6129d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:37.421619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b4263c17-b93c-4b84-ab95-c8a4cf6129d0 None None] MFA Rules not processed for user `37e7fecdaae94842b6917152fb7f1598`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:37.423918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b4263c17-b93c-4b84-ab95-c8a4cf6129d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:37.655015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1303/2595] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:38.466184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:38.468516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:38.542932 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:38.545934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:38.546212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:38.552291 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:38.590331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] MFA Rules not processed for user `d3eebc82ce654013998f58585c123a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:38.592170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:38.618545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f322b-98d0-405d-a295-97bcb36d729b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:38.631939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:38.633853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:38.634632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:38.635648 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:38.636148 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:38.639368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:38.657051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:38.657641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:38.682442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1293/2596] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:38.768272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1304/2597] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:38.771766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:38.774930 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:38.985718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d46-a398-4a89-ab3e-66e84d029fbe service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.004738 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.006379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.007413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.007854 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.009797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.013451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.060560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1294/2598] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:30:38 2022] POST /identity/v3/projects => generated 379 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:30:39.066194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:39.079745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb4bba-353a-48c2-aee4-44442067ab64 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.187218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.188373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.188907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.190933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.191524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.198728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.238832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1305/2599] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:30:39 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:30:39.240076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:39.276335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6fb9-b9d4-48ff-afbe-ab8b51c3dced admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.292125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.294557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.294834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.295110 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.295968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.299225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.355763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1295/2600] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:30:39.361861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:39.427166 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f742cb-4241-4837-87b6-f26c076d362b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.441053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.442765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.443350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.443350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] PATH_INFO: `/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e/users/15200f16c51e42d59351b04a311797df/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.501397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5a80ea9f1b43d88e4fe96b8b87716e, user_id=15200f16c51e42d59351b04a311797df, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.506434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.549190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1306/2601] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:30:39 2022] PUT /identity/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e/users/15200f16c51e42d59351b04a311797df/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:39.554710 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:39.579656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860fe4a-320d-4b5b-86a1-26a79b1d5fba admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.716010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.717027 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.717476 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.717887 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.718803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.721034 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384709f-6bb4-4766-bcab-5708b9cab347 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.742227 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1296/2602] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:30:39.746629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:39.761309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f59be-8a66-42b0-b2d6-75965c96b416 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:39.781622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:39.783520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:39.784096 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:39.784775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] PATH_INFO: `/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e/users/15200f16c51e42d59351b04a311797df/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:39.791639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5a80ea9f1b43d88e4fe96b8b87716e, user_id=15200f16c51e42d59351b04a311797df, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:39.797396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:39.833403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1307/2603] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:30:39 2022] PUT /identity/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e/users/15200f16c51e42d59351b04a311797df/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:30:49.730162 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:49.825554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:30:52.053774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:52.054547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:52.054547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:52.116880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] MFA Rules not processed for user `15200f16c51e42d59351b04a311797df`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:30:52.121667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:52.273190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1297/2604] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:30:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:30:52.293206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:52.296339 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:52.440952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d76df-fad2-4359-b7b6-c705ff4805c2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:52.449750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:52.450646 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:52.451004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:52.451331 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:52.598730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:52.602287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5da1c-ebfd-4fd2-8093-f22a8834305f service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:52.624511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1308/2605] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:30:54.026951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:30:54.056430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360fc9a-ab3b-49e0-ba07-5548a1089e8c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:30:54.072409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:30:54.073674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:30:54.074167 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:30:54.074649 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:30:54.083198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:30:54.116575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:30:54.121202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:30:54.172168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1298/2606] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:30:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:02.611070 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=56 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:04.137970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=40 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:10.241143 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:10.242076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:10.242076 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:10.275403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] MFA Rules not processed for user `d3eebc82ce654013998f58585c123a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:10.277378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:10.288242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1309/2607] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:18.455315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:18.547842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bfbed-9189-4d09-a85a-71afffa47dc8 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:18.552307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:18.553074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:18.553451 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:18.553773 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:18.556095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:18.638641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:18.640239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:18.652937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1299/2608] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:20.265998 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:23.500619 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:23.507150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:23.672269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:23.675210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:23.697136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94c5bc-e5ed-4cba-9a46-f3b9305f1599 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:23.704389 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df3a4e-9fae-43a9-9e33-f17fe76ebda0 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:23.711147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:23.712311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:23.712756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:23.713400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] PATH_INFO: `/v3/users/644c0a532fa747c7abf4d53759e89181` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:23.720574 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644c0a532fa747c7abf4d53759e89181)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:23.723550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:23.724865 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:23.725457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:23.726112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:23.726689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:23.732071 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:23.765823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:23.769707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:23.802458 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1310/2609] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:23.855828 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644c0a532fa747c7abf4d53759e89181'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:23.872845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644c0a532fa747c7abf4d53759e89181'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:23.885261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644c0a532fa747c7abf4d53759e89181'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:23.904983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644c0a532fa747c7abf4d53759e89181'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:23.906156 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ff52f12b-8d22-46c9-a7d4-655591a82347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644c0a532fa747c7abf4d53759e89181'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:23.914923 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1300/2610] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:23 2022] DELETE /identity/v3/users/644c0a532fa747c7abf4d53759e89181 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:24.180917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.195363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5acbb54-d86e-4e46-8e28-7de75e605ba2 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.274995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.275886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.276167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.276439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] PATH_INFO: `/v3/projects/5c4be8083168475198c66cca890b18a9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.277819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4be8083168475198c66cca890b18a9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.280050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.360241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4be8083168475198c66cca890b18a9', 'request_id': 'req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:24.372593 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4be8083168475198c66cca890b18a9', 'request_id': 'req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:24.385557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4be8083168475198c66cca890b18a9', 'request_id': 'req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:24.407786 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] The token cache is being invalidate because project 5c4be8083168475198c66cca890b18a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:24.408152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:24.408943 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2a85cd-03a5-4776-b660-9b8505f4b9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:24.412225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1311/2611] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:24 2022] DELETE /identity/v3/projects/5c4be8083168475198c66cca890b18a9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:24.419202 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.419878 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.420448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.463061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:24.465920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:24.504514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:24.575263 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1301/2612] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:24.586194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.587056 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.587442 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.610787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:24.612473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:24.620818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1312/2613] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:24.625835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.636654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb555b-eacc-4d03-9726-9af53dd3d174 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.641778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.642909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.643387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.643740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.644575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:31:24.644950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.646807 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.665024 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1302/2614] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:31:24.666669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.677032 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da978c71-4e81-4d78-9fdf-73b56a697943 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.683052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.683750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.683750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.684398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.685039 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.687326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.704313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:24.727758 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1313/2615] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:24 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:24.732811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.746153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488030f0-39ab-44d4-8ae7-2bfd3815ddc5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.826305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.827364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.827835 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.828308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.829629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.833500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.868676 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1303/2616] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:24 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:24.873838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.888528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a13686-fca0-454d-b841-0094effbc44c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.895787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.896579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.896905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.897280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.898256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.900659 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.915038 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1314/2617] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:24.920661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.935796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5970d-69d6-4fc2-af08-63cb295e4b10 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.942709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.943960 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.944509 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.945081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:24.946045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:24.947947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31c815-651c-4b2d-92f2-deaebac98628 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:24.964361 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1304/2618] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:24.969049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:24.981231 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad736a4-5904-4543-961a-b1e1fde99960 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:24.988763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:24.990006 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:24.990383 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:24.990829 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] PATH_INFO: `/v3/projects/bbe5647b049e427ebf2d19033386321f/users/a0116d75a38449ad8b890203c12cc079/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.012999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe5647b049e427ebf2d19033386321f, user_id=a0116d75a38449ad8b890203c12cc079, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.016527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.034516 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1315/2619] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:24 2022] PUT /identity/v3/projects/bbe5647b049e427ebf2d19033386321f/users/a0116d75a38449ad8b890203c12cc079/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:25.049957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.050498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.051082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.097038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] MFA Rules not processed for user `a0116d75a38449ad8b890203c12cc079`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:25.098388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:25.192223 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1305/2620] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:25.199016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.308801 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7403d9-a05b-47b9-ad6e-fd80f0138680 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.314194 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.314911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.315191 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.315469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.316173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.318215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.349696 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1316/2621] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:25 2022] POST /identity/v3/projects => generated 393 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:25.354543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.367207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e08495-9b07-4af9-890d-91f4cdd04879 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.382913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:25.445969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.446787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.447117 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.447436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.448305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.450443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.482633 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1306/2622] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:25 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:25.484309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.496840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd97-cb29-4dd5-b64b-e61fdf960fb7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.502898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.503651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.503935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.504219 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.505007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.507346 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.520514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1317/2623] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:25.526461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.539047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b35000-dfa3-4fb3-9d5e-fa4f702bee07 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.545808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.546882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.547330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.547816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] PATH_INFO: `/v3/projects/f834f8e1229343738a36355cf3f6fd93/users/56ad2fd405a140feb19f4667c201fec7/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.568961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f834f8e1229343738a36355cf3f6fd93, user_id=56ad2fd405a140feb19f4667c201fec7, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.571958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.588963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1307/2624] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:25 2022] PUT /identity/v3/projects/f834f8e1229343738a36355cf3f6fd93/users/56ad2fd405a140feb19f4667c201fec7/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:25.593569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.605203 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3d5a0-bd20-4bb7-88de-51e81136fa87 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.698291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.699725 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.701867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.701867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.701867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.703926 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785036a-9cf2-4256-be94-ea187f4dcb91 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.721449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1318/2625] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:25.723858 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:25.740483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c6fa0-12fc-4da5-9a5d-0e083eddecaa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:25.748906 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:25.750100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.750505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.750919 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] PATH_INFO: `/v3/projects/f834f8e1229343738a36355cf3f6fd93/users/56ad2fd405a140feb19f4667c201fec7/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.754492 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f834f8e1229343738a36355cf3f6fd93, user_id=56ad2fd405a140feb19f4667c201fec7, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:25.757278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:25.779056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1308/2626] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:25 2022] PUT /identity/v3/projects/f834f8e1229343738a36355cf3f6fd93/users/56ad2fd405a140feb19f4667c201fec7/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:25.793441 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:25.793951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:25.794434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:25.835160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] MFA Rules not processed for user `56ad2fd405a140feb19f4667c201fec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:25.837569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:25.863244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:25.985125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1319/2627] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:25.997935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:26.000279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:26.124487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726797fc-e744-4f85-b860-0663ad374ee9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:26.130347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:26.131716 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:26.131716 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:26.132043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:26.149125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:26.152003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:26.180179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1309/2628] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:26.255397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:26.259202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:26.276248 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49337b02-b9a7-410e-8513-66452b97bfcf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:26.282247 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:26.283941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:26.284805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:26.285532 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:26.371974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:26.374015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:26.389202 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1320/2629] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:29.139779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:29.141761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:29.222208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c10a7e-02f9-4a51-8d0d-ca4927fa9b37 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:29.227003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:29.227683 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:29.227964 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:29.228240 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:29.230808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:29.238856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:29.319725 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:29.321208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ea9b7-ccc3-48c3-9504-040d9ab3d4e1 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:29.333726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1310/2630] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:30.248136 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:30.343608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e909866-7567-4e8f-97bb-02b9825afe9e service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:30.351783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:30.353498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:30.353988 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:30.354466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] PATH_INFO: `/v3/users/a0116d75a38449ad8b890203c12cc079` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:30.356770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0116d75a38449ad8b890203c12cc079)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:30.360524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:30.445914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0116d75a38449ad8b890203c12cc079'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:30.452972 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:30.463605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0116d75a38449ad8b890203c12cc079'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:30.465819 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0116d75a38449ad8b890203c12cc079'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:30.480227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0116d75a38449ad8b890203c12cc079'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:30.482049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0116d75a38449ad8b890203c12cc079'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:30.515002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1321/2631] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:30 2022] DELETE /identity/v3/users/a0116d75a38449ad8b890203c12cc079 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:30.521463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:30.522366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:30.522366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:30.546145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:30.547576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:30.639208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1311/2632] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:30.650784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:30.652971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:30.731718 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263853a5-6952-4a04-89ad-3240620b1bfc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:30.736373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:30.737504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:30.737785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:30.738081 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:30.750611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:30.753064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:30.767131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1322/2633] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:31.234951 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:31.237559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:31.265802 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09f3d6-50b8-4c0e-8b9b-bd96e8a4ff79 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:31.271545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:31.272713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:31.273018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:31.273302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] PATH_INFO: `/v3/projects/bbe5647b049e427ebf2d19033386321f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:31.276466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe5647b049e427ebf2d19033386321f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:31.279458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:31.363221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe5647b049e427ebf2d19033386321f', 'request_id': 'req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.374646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe5647b049e427ebf2d19033386321f', 'request_id': 'req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.382344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe5647b049e427ebf2d19033386321f', 'request_id': 'req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.396125 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] The token cache is being invalidate because project bbe5647b049e427ebf2d19033386321f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:31.396413 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.397079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.398839 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1312/2634] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:31 2022] DELETE /identity/v3/projects/bbe5647b049e427ebf2d19033386321f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:31.404077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:31.415830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873624a-a119-4baa-a527-333e0dc7d01b service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:31.495448 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:31.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:31.497077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:31.497537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] PATH_INFO: `/v3/users/56ad2fd405a140feb19f4667c201fec7` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:31.498721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ad2fd405a140feb19f4667c201fec7)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:31.500776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:31.577797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ad2fd405a140feb19f4667c201fec7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.598252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ad2fd405a140feb19f4667c201fec7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.599689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ad2fd405a140feb19f4667c201fec7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.618673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ad2fd405a140feb19f4667c201fec7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.619795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9c198bfe-d8f1-4d60-9912-80d73e24cd7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ad2fd405a140feb19f4667c201fec7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:31.635909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:31.661262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1323/2635] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:31 2022] DELETE /identity/v3/users/56ad2fd405a140feb19f4667c201fec7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:32.149006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:32.151863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:32.168433 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502ced4-90c1-4b3e-84d8-ddcac02ffb6f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:32.217561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:32.269467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:32.270432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.270781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.271059 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] PATH_INFO: `/v3/projects/f834f8e1229343738a36355cf3f6fd93` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.272629 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f834f8e1229343738a36355cf3f6fd93)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:32.275120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:32.358797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f834f8e1229343738a36355cf3f6fd93', 'request_id': 'req-9aad29cf-b07e-4ead-9dab-df1d863b5fef'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:32.377299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f834f8e1229343738a36355cf3f6fd93', 'request_id': 'req-9aad29cf-b07e-4ead-9dab-df1d863b5fef'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:32.389098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f834f8e1229343738a36355cf3f6fd93', 'request_id': 'req-9aad29cf-b07e-4ead-9dab-df1d863b5fef'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:32.414613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] The token cache is being invalidate because project f834f8e1229343738a36355cf3f6fd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:32.414933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:32.415957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9aad29cf-b07e-4ead-9dab-df1d863b5fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:32.418793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1313/2636] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:32 2022] DELETE /identity/v3/projects/f834f8e1229343738a36355cf3f6fd93 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:32.427286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.428967 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.429948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.481952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:32.483288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:32.616347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1324/2637] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:32.627160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410042b-4540-402c-b612-d9566d038fae None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.627698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410042b-4540-402c-b612-d9566d038fae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.628185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410042b-4540-402c-b612-d9566d038fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.658338 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2410042b-4540-402c-b612-d9566d038fae None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:32.660546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2410042b-4540-402c-b612-d9566d038fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:32.676139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1314/2638] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:32.682179 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:32.695392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07139d55-38ff-4e42-8395-a39dca520950 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:32.701571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:32.702767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.703246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.703747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.705208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:31:32.705765 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:32.712687 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:32.732782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1325/2639] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 16:31:32.739653 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410042b-4540-402c-b612-d9566d038fae None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:32.752631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410042b-4540-402c-b612-d9566d038fae None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:32.760250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:32.761415 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.761898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.762387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.763684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:32.767513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:32.816100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1315/2640] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:32 2022] POST /identity/v3/projects => generated 399 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:32.820517 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:32.835641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01e92e-3724-45c2-8f21-d476dbf89172 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:32.927600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:32.928678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.929268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.929645 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.930599 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:32.932885 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:32.967604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1326/2641] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:32 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:32.972912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:32.987654 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603ac72-6f6a-4c75-9e76-10261542ee82 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:32.993934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:32.994745 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:32.995063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:32.995404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:32.996323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:32.998328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.012922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1316/2642] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:31:33.017957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.029835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d1e8e-456e-490e-9e22-5bfb4b68135f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.035850 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.036901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.037407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.037841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.038799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.041054 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fcfd-5d19-4fbd-a3b9-b6849dbbbf46 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.057274 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1327/2643] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:33.062848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.075739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adafcd1-cf06-4e18-a7c0-69adf605d205 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.082996 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.084483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.084904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.085325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] PATH_INFO: `/v3/projects/e91bda86c7654c0f867c2846b6596c2d/users/58c099d5d0b94232b15502019e554540/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.113573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91bda86c7654c0f867c2846b6596c2d, user_id=58c099d5d0b94232b15502019e554540, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.116445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.133983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1317/2644] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:33 2022] PUT /identity/v3/projects/e91bda86c7654c0f867c2846b6596c2d/users/58c099d5d0b94232b15502019e554540/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:33.146397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.146711 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.146951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.189092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] MFA Rules not processed for user `58c099d5d0b94232b15502019e554540`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:33.191362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:33.268037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:33.295602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1328/2645] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:31:33.301534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.396673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:33.416905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc40c-94c1-4c5b-b8bf-5af57eb62a9d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.425443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.426519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.426950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.427396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.428697 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.432421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.478502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1318/2646] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:33 2022] POST /identity/v3/projects => generated 399 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:31:33.483104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.495252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe0975-7db2-40b3-8f4f-a3b4f3bd4d90 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.561316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.562059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.562469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.562666 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.563376 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.565472 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.590864 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1329/2647] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:33 2022] POST /identity/v3/users => generated 306 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:33.595505 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.607518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d171dc0-1b34-4cc0-b7d6-e41343297756 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.613369 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.614092 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.614375 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.614646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.615396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.617063 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.631013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1319/2648] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:33.636105 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.647891 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb40d5-dce1-4cd9-8cbd-649b6e8f46e8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.654108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.655047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.655328 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.655594 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] PATH_INFO: `/v3/projects/f849b2408e3d4bc8b53795aa1a296e52/users/c0b2b6e2d28448f5bac6baa0e848375f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.674959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f849b2408e3d4bc8b53795aa1a296e52, user_id=c0b2b6e2d28448f5bac6baa0e848375f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.677804 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.694794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1330/2649] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:33 2022] PUT /identity/v3/projects/f849b2408e3d4bc8b53795aa1a296e52/users/c0b2b6e2d28448f5bac6baa0e848375f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:33.698975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.712087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ccb63-2e08-48e5-8e88-8af800424c44 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.783081 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.783797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.784102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.784429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.785191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.787021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f9046-fe8c-43c7-b24d-5cfb05f33309 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.804566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1320/2650] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:33.809680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:33.823566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b4ce-12bb-4601-a0ed-8d6cec7d60d2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:33.828983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:33.830024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.830322 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.830600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] PATH_INFO: `/v3/projects/f849b2408e3d4bc8b53795aa1a296e52/users/c0b2b6e2d28448f5bac6baa0e848375f/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.832930 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f849b2408e3d4bc8b53795aa1a296e52, user_id=c0b2b6e2d28448f5bac6baa0e848375f, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:33.835809 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:33.848953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1331/2651] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:33 2022] PUT /identity/v3/projects/f849b2408e3d4bc8b53795aa1a296e52/users/c0b2b6e2d28448f5bac6baa0e848375f/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:33.867562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:33.868116 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:33.868502 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:33.903151 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] MFA Rules not processed for user `c0b2b6e2d28448f5bac6baa0e848375f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:33.904448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:33.996276 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1321/2652] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:33 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:34.009065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:34.011557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:34.096960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850f138-5108-4be7-b6ee-ed3eb09466b2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:34.103257 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:34.104335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:34.104742 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:34.105172 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:34.122764 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:34.125503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:34.144579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1332/2653] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:34.241797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:34.252804 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65924-f517-42af-b4c8-1ce0300a9997 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:34.257074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:34.257922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:34.258262 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:34.258582 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:34.318395 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:34.336464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:34.338182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84147df2-fd0b-4a04-940b-e531cc8fd375 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:34.350353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1322/2654] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:38.943282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:39.002613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:39.026820 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae05a4-487d-417f-b296-bf0f7fecd4c4 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:39.031969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:39.032953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:39.033225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:39.033524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] PATH_INFO: `/v3/users/58c099d5d0b94232b15502019e554540` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:39.034876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c099d5d0b94232b15502019e554540)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:39.036860 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:39.098218 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c099d5d0b94232b15502019e554540'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.109473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c099d5d0b94232b15502019e554540'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.110227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c099d5d0b94232b15502019e554540'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.120999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c099d5d0b94232b15502019e554540'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.122180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c099d5d0b94232b15502019e554540'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.147216 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1333/2655] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:38 2022] DELETE /identity/v3/users/58c099d5d0b94232b15502019e554540 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:39.152241 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:39.153135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:39.153135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:39.176953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:39.178467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:39.272253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1323/2656] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:39.288084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:39.290553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:39.391313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719255b-f3fe-406d-a176-8b933e9f52e6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:39.395538 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:39.396217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:39.396494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:39.396769 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:39.407449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:39.408937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:39.420831 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1334/2657] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:39.795527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:39.797958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:39.820899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0b64e-73ed-4873-955f-0465c5bc26f8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:39.830326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:39.831465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:39.831903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:39.832340 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] PATH_INFO: `/v3/projects/e91bda86c7654c0f867c2846b6596c2d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:39.834026 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91bda86c7654c0f867c2846b6596c2d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:39.836685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:39.903442 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91bda86c7654c0f867c2846b6596c2d', 'request_id': 'req-e3bb46d1-2a54-4d53-a108-776b522b31d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.913452 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91bda86c7654c0f867c2846b6596c2d', 'request_id': 'req-e3bb46d1-2a54-4d53-a108-776b522b31d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.920829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91bda86c7654c0f867c2846b6596c2d', 'request_id': 'req-e3bb46d1-2a54-4d53-a108-776b522b31d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.935982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] The token cache is being invalidate because project e91bda86c7654c0f867c2846b6596c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:39.936356 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.937315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:39.939216 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1324/2658] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:39 2022] DELETE /identity/v3/projects/e91bda86c7654c0f867c2846b6596c2d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:39.944602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:39.956939 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253fd49-336c-403a-8efe-cf57d0cf6ecd service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:40.038881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:40.039827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:40.040193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:40.040510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] PATH_INFO: `/v3/users/c0b2b6e2d28448f5bac6baa0e848375f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:40.041997 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b2b6e2d28448f5bac6baa0e848375f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:40.044053 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:40.050283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:40.124779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b2b6e2d28448f5bac6baa0e848375f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.139395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b2b6e2d28448f5bac6baa0e848375f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.140603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b2b6e2d28448f5bac6baa0e848375f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.152568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b2b6e2d28448f5bac6baa0e848375f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.153596 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-59075615-da2c-41f1-83f5-77514722faff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b2b6e2d28448f5bac6baa0e848375f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.184744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1335/2659] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:39 2022] DELETE /identity/v3/users/c0b2b6e2d28448f5bac6baa0e848375f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:40.586065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:40.587999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:40.622325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb46d1-2a54-4d53-a108-776b522b31d1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:40.707701 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:40.708936 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:40.709290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:40.709767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] PATH_INFO: `/v3/projects/f849b2408e3d4bc8b53795aa1a296e52` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:40.711355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f849b2408e3d4bc8b53795aa1a296e52)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:40.713405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:40.731054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:40.789594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f849b2408e3d4bc8b53795aa1a296e52', 'request_id': 'req-bb113956-2659-44a3-b020-0657756c788b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.800350 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f849b2408e3d4bc8b53795aa1a296e52', 'request_id': 'req-bb113956-2659-44a3-b020-0657756c788b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.811474 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f849b2408e3d4bc8b53795aa1a296e52', 'request_id': 'req-bb113956-2659-44a3-b020-0657756c788b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.833222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] The token cache is being invalidate because project f849b2408e3d4bc8b53795aa1a296e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:40.833751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.834852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bb113956-2659-44a3-b020-0657756c788b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:40.837724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1325/2660] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:40 2022] DELETE /identity/v3/projects/f849b2408e3d4bc8b53795aa1a296e52 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:40.845260 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:40.845798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:40.846299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:40.875853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:40.878055 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:40.960017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1336/2661] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:40.971596 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:40.972120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:40.972657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:40.998041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:40.999295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:41.008208 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1326/2662] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:41.013174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.024211 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab964a-33b0-48d4-ba71-ca951b8ee828 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.029518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.030230 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.030529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.030772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.031678 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:31:41.031984 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.033706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.048299 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1337/2663] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:31:41.053458 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.063937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc1c99-123c-4fb9-8aad-48d71267da66 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.069245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.070047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.070484 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.070808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.071740 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.074132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.110909 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1327/2664] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:41 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:41.118925 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.131002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b2f0d-7ce6-492b-9758-3f01b2a6674b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.210707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.211826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.212310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.212755 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.214061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.217826 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.244966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:41.252938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1338/2665] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:41 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:41.258767 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.272056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846163-b7c5-443b-9f07-2a3c9b3d7ce5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.278913 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.279813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.280152 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.280468 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.281302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.283337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.297843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1328/2666] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:41.303234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.315349 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d398-2e19-4530-a3eb-adff837a9cc4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.321855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.322636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.322935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.323229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.324057 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.326013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad6424-c1b6-4a98-9ac7-bf92728b3b49 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.342920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1339/2667] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:41.350107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.365508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be56-c35e-4e1e-b912-cdbc3b543216 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.371153 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.372077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.372368 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.372637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] PATH_INFO: `/v3/projects/a867706b94ae49eb93daade9cf44d25c/users/ee50df24b9bd482b9c24852cf9be4aea/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.391344 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a867706b94ae49eb93daade9cf44d25c, user_id=ee50df24b9bd482b9c24852cf9be4aea, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.394469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.411790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1329/2668] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:41 2022] PUT /identity/v3/projects/a867706b94ae49eb93daade9cf44d25c/users/ee50df24b9bd482b9c24852cf9be4aea/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:41.423763 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.424112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.424597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.468823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] MFA Rules not processed for user `ee50df24b9bd482b9c24852cf9be4aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:41.471002 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:41.565139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1340/2669] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:41.570598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.649077 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21f01e-00c4-4498-8f79-db57985d409d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.655165 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.655933 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.656335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.656762 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.657670 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.659830 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.699566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1330/2670] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:41 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:41.706556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.727991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae4aa4-36f1-4686-93bb-603127761b66 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.812292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.813029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.813319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.813756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.814585 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.816747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.845256 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1341/2671] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:41 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:41.851122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.871559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4c9bc-093a-4221-b27e-3a5a4cba857b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.878425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.879269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.879598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.880104 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.881097 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.883095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.896968 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1331/2672] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:41.901807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.913836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933f331-e579-48e8-b366-a0c7c9986b04 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.919792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.920539 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.920858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.921197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.922120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.924040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731431af-b10d-46cb-b387-6b184cb8cc8d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:41.936913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1342/2673] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:31:41.942678 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:41.955631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ae65-914e-4943-a428-8241e39715fa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:41.962036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:41.963067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:41.963347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:41.963741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] PATH_INFO: `/v3/projects/1991f9565f7d43c0a514dfd1e746f619/users/16f032c84cd7457d8c4fc4b69b3bc460/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:41.974693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:41.985724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1991f9565f7d43c0a514dfd1e746f619, user_id=16f032c84cd7457d8c4fc4b69b3bc460, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:41.991161 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.009739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1332/2674] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:41 2022] PUT /identity/v3/projects/1991f9565f7d43c0a514dfd1e746f619/users/16f032c84cd7457d8c4fc4b69b3bc460/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:42.022767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.023429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.023600 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.061388 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] MFA Rules not processed for user `16f032c84cd7457d8c4fc4b69b3bc460`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:42.062800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:42.127208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:42.159210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1343/2675] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:42.164563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.254917 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc307c-6b5a-4fe9-8b42-833c5e623243 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.261135 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.261916 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.262166 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.262542 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.263175 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.266295 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.306357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1333/2676] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:42 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:42.311169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.327307 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a09c3-7b85-4874-8ce7-975d0fe7f034 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.426582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.427210 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.427498 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.427774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.428483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.430508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.457398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1344/2677] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:42 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:42.463991 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.489982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa2cdc-6b73-4639-93ab-68ee9590c679 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.498192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.499467 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.500006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.500541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.501958 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.505055 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.526219 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1334/2678] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:31:42.532127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.555440 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95414d9c-3193-4a39-9d62-ea87804a2426 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.563365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.564569 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.565052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.565432 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] PATH_INFO: `/v3/projects/94d1debe88704cadbfa0b106a9b118e1/users/f041b4beae5f46aa94354e2b1e138df0/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.592018 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d1debe88704cadbfa0b106a9b118e1, user_id=f041b4beae5f46aa94354e2b1e138df0, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.597137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.624975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1345/2679] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:42 2022] PUT /identity/v3/projects/94d1debe88704cadbfa0b106a9b118e1/users/f041b4beae5f46aa94354e2b1e138df0/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:42.631339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.652521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486a04f-fac3-4ff3-931c-b70dbf070510 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.759114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.759942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.760279 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.760690 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.761766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.763735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a4b90-c378-440c-8c25-49180c9d19e8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.779080 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1335/2680] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:42.784849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:42.796965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925488e-ea99-4da6-863a-f675f5201b83 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:42.802445 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:42.803357 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.803636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.803905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] PATH_INFO: `/v3/projects/94d1debe88704cadbfa0b106a9b118e1/users/f041b4beae5f46aa94354e2b1e138df0/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.806124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d1debe88704cadbfa0b106a9b118e1, user_id=f041b4beae5f46aa94354e2b1e138df0, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:42.809727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:42.825095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1346/2681] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:42 2022] PUT /identity/v3/projects/94d1debe88704cadbfa0b106a9b118e1/users/f041b4beae5f46aa94354e2b1e138df0/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:42.845180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:42.845838 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:42.846271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:42.888207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] MFA Rules not processed for user `f041b4beae5f46aa94354e2b1e138df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:42.889501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:42.986414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:43.034422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1336/2682] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:43.051884 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:43.054430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:43.145991 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f8949-df75-45c6-b665-78a1784e06b3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:43.152409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:43.153527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:43.153812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:43.154124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:43.166507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:43.168173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:43.187434 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1347/2683] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:43.243872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:43.245586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:43.260266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64602680-9ed8-47db-bd3d-e8d1afa5976f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:43.264684 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:43.265464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:43.265749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:43.266068 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:43.344454 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:43.346014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65022c3-e9ba-44dc-b49f-aee63ef34b1b service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:43.358785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1337/2684] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:46.735981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:46.799373 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:46.827776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29b13-a456-4d78-91dc-01cfdbd720b2 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:46.836224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:46.838728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:46.839010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:46.839273 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] PATH_INFO: `/v3/users/ee50df24b9bd482b9c24852cf9be4aea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:46.840825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee50df24b9bd482b9c24852cf9be4aea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:46.843010 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:46.912700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee50df24b9bd482b9c24852cf9be4aea'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:46.927220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee50df24b9bd482b9c24852cf9be4aea'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:46.928381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee50df24b9bd482b9c24852cf9be4aea'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:46.941695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee50df24b9bd482b9c24852cf9be4aea'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:46.942747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee50df24b9bd482b9c24852cf9be4aea'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:46.977914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1348/2685] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:46 2022] DELETE /identity/v3/users/ee50df24b9bd482b9c24852cf9be4aea => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:46.983864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:46.984782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:46.984782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:47.012819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:47.015108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:47.097884 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1338/2686] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:47.110222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:47.112392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:47.184029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0ab13-df28-48f0-8b3d-0abeded38be5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:47.188300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:47.189118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:47.189564 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:47.189933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:47.200800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:47.202452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:47.214414 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1349/2687] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:47.585172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:47.587339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:47.607551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ea633-2189-4eb4-b625-3f0b7c0a7c33 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:47.615047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:47.616189 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:47.616580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:47.617082 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] PATH_INFO: `/v3/projects/a867706b94ae49eb93daade9cf44d25c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:47.619056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a867706b94ae49eb93daade9cf44d25c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:47.621749 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:47.697043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a867706b94ae49eb93daade9cf44d25c', 'request_id': 'req-a7051144-2354-46a4-a2de-6bcf051bc5e4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.708103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a867706b94ae49eb93daade9cf44d25c', 'request_id': 'req-a7051144-2354-46a4-a2de-6bcf051bc5e4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.716164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a867706b94ae49eb93daade9cf44d25c', 'request_id': 'req-a7051144-2354-46a4-a2de-6bcf051bc5e4'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.731329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] The token cache is being invalidate because project a867706b94ae49eb93daade9cf44d25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:47.731659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.732447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.737786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1339/2688] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:47 2022] DELETE /identity/v3/projects/a867706b94ae49eb93daade9cf44d25c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:47.739215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:47.758129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58932-e6b3-4ad7-811b-aafa3e5447c1 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:47.855677 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:47.856669 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:47.856955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:47.857235 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] PATH_INFO: `/v3/users/16f032c84cd7457d8c4fc4b69b3bc460` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:47.858597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f032c84cd7457d8c4fc4b69b3bc460)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:47.860716 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:47.867995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:47.950140 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f032c84cd7457d8c4fc4b69b3bc460'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.960389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f032c84cd7457d8c4fc4b69b3bc460'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.961501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f032c84cd7457d8c4fc4b69b3bc460'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.975582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f032c84cd7457d8c4fc4b69b3bc460'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.976288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f032c84cd7457d8c4fc4b69b3bc460'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:47.998314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1350/2689] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:47 2022] DELETE /identity/v3/users/16f032c84cd7457d8c4fc4b69b3bc460 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:48.455265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:48.457371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:48.473631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:48.484647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051144-2354-46a4-a2de-6bcf051bc5e4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:48.596391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:48.598072 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:48.598587 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:48.599070 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] PATH_INFO: `/v3/projects/1991f9565f7d43c0a514dfd1e746f619` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:48.601457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1991f9565f7d43c0a514dfd1e746f619)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:48.605260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:48.693898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1991f9565f7d43c0a514dfd1e746f619', 'request_id': 'req-dbc73514-542b-4ab0-963e-867722b78726'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.704528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1991f9565f7d43c0a514dfd1e746f619', 'request_id': 'req-dbc73514-542b-4ab0-963e-867722b78726'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.713213 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1991f9565f7d43c0a514dfd1e746f619', 'request_id': 'req-dbc73514-542b-4ab0-963e-867722b78726'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.727978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] The token cache is being invalidate because project 1991f9565f7d43c0a514dfd1e746f619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:48.728296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.729099 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.730992 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1340/2690] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:48 2022] DELETE /identity/v3/projects/1991f9565f7d43c0a514dfd1e746f619 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:48.735879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:48.738444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:48.755746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9c487-6580-4bac-8914-0451088b4b67 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:48.834921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:48.836459 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:48.836951 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:48.837494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] PATH_INFO: `/v3/users/f041b4beae5f46aa94354e2b1e138df0` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:48.839704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f041b4beae5f46aa94354e2b1e138df0)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:48.843463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:48.937241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f041b4beae5f46aa94354e2b1e138df0'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.952582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f041b4beae5f46aa94354e2b1e138df0'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.953744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f041b4beae5f46aa94354e2b1e138df0'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.966694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f041b4beae5f46aa94354e2b1e138df0'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:48.967823 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ba05a2a1-a1bc-4b29-a102-1447104797af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f041b4beae5f46aa94354e2b1e138df0'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.000900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1351/2691] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:48 2022] DELETE /identity/v3/users/f041b4beae5f46aa94354e2b1e138df0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:49.418270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:49.420302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:49.436623 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73514-542b-4ab0-963e-867722b78726 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:49.544508 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:49.545845 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:49.546238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:49.546620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] PATH_INFO: `/v3/projects/94d1debe88704cadbfa0b106a9b118e1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:49.548293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94d1debe88704cadbfa0b106a9b118e1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:49.550734 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:49.623602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:49.638947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94d1debe88704cadbfa0b106a9b118e1', 'request_id': 'req-2cca23df-0599-4ae1-bdda-1c87b48d8043'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.651857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94d1debe88704cadbfa0b106a9b118e1', 'request_id': 'req-2cca23df-0599-4ae1-bdda-1c87b48d8043'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.659971 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94d1debe88704cadbfa0b106a9b118e1', 'request_id': 'req-2cca23df-0599-4ae1-bdda-1c87b48d8043'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.674463 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] The token cache is being invalidate because project 94d1debe88704cadbfa0b106a9b118e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:49.674814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.676278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2cca23df-0599-4ae1-bdda-1c87b48d8043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:49.677785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1341/2692] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:49 2022] DELETE /identity/v3/projects/94d1debe88704cadbfa0b106a9b118e1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:49.687306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:49.687770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:49.688294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:49.701605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:49.727840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:49.729723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:49.816832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1352/2693] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:49.827979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:49.828488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:49.828962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:49.858269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:49.859568 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:49.869477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1342/2694] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:49.874602 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:49.885851 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ee056-e635-4a47-8882-f25840fbb459 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:49.891288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:49.892017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:49.892306 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:49.892584 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:49.893649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:31:49.893981 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:49.895960 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:49.912129 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1353/2695] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:31:49.917501 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:49.928322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed929-ee22-4131-923f-2e5d354e5d98 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:49.934674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:49.935366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:49.935666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:49.935941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:49.936686 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:49.938848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:49.982122 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1343/2696] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:49 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:49.988554 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.007152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171d85-35ff-4006-9eca-7e196e3560fd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.095574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.096653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.097150 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.097682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.098975 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.102874 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.137712 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1354/2697] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:49 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:50.144598 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.163331 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242b5b-534d-4e0c-8e81-328f95a6d467 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.171655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.172852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.173271 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.173585 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.174391 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.176066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.200337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1344/2698] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:50.204882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.223199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b655bd-f36d-4f12-bdf3-5acd2622ecd4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.230293 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.231016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.231302 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.231579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.232462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.234411 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a7987-607b-4dc3-a027-47d1fa0de27f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.248083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1355/2699] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:31:50.254349 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.268840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687aa0e-af40-4021-aaea-fd988e138234 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.274760 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.276008 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.276438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.277006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] PATH_INFO: `/v3/projects/b987cd8f98aa490db88a846caeee7212/users/d109f2d02ab4459991da612505922461/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.312679 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b987cd8f98aa490db88a846caeee7212, user_id=d109f2d02ab4459991da612505922461, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.318293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.341224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1345/2700] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:50 2022] PUT /identity/v3/projects/b987cd8f98aa490db88a846caeee7212/users/d109f2d02ab4459991da612505922461/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:50.357023 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.357381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.357799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.395419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] MFA Rules not processed for user `d109f2d02ab4459991da612505922461`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:50.397040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:50.502361 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:50.512238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1356/2701] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:50.517410 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.602441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07573d54-6934-4559-90d1-4e2fd79f7db8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.608494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.609260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.609673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.610065 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.611061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.614700 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.665620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1346/2702] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:50 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:50.667524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.694064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6d19-9815-40ea-a21f-0cc579f81057 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.803238 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.803948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.804363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.805005 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.805814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.808176 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.838199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1357/2703] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:50 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:31:50.843727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.863798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187cbc7-ca0d-4826-8b25-69fffe82cd5e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.869728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.870720 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.871000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.871301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.872303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.874543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.889843 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1347/2704] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:50.894814 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.908941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f6160-7746-4ee8-b615-ba8b729d8c9e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:50.917415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:50.919094 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:50.919611 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:50.920093 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] PATH_INFO: `/v3/projects/dff63980e5294b57a803118aea20d68e/users/4267aa21b3f44cb98469638b56c87659/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:50.946173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff63980e5294b57a803118aea20d68e, user_id=4267aa21b3f44cb98469638b56c87659, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:50.949061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:50.971397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1358/2705] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:50 2022] PUT /identity/v3/projects/dff63980e5294b57a803118aea20d68e/users/4267aa21b3f44cb98469638b56c87659/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:50.976680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:50.990957 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083a7f-a01f-413d-8098-2dfc0231a47e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:51.037186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:51.081895 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:51.083052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:51.083624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:51.084108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:51.085487 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:51.088486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d9cbe-69ed-465d-bc3d-894700f44f5b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:51.108641 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1348/2706] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:51.114729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:51.133830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eaaf4-53c8-46d3-b415-c04d6153d2f8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:51.142473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:51.144013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:51.144506 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:51.144976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] PATH_INFO: `/v3/projects/dff63980e5294b57a803118aea20d68e/users/4267aa21b3f44cb98469638b56c87659/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:51.148333 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff63980e5294b57a803118aea20d68e, user_id=4267aa21b3f44cb98469638b56c87659, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:51.153483 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:51.173217 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1359/2707] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:51 2022] PUT /identity/v3/projects/dff63980e5294b57a803118aea20d68e/users/4267aa21b3f44cb98469638b56c87659/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:51.186402 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:51.186882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:51.187358 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:51.228974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] MFA Rules not processed for user `4267aa21b3f44cb98469638b56c87659`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:51.230798 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:51.342793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1349/2708] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:51.358127 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:51.360900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:51.471798 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541265e-27a6-4885-9ee3-ea970e6e0b91 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:51.479842 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:51.480920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:51.481462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:51.481882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:51.500465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:51.504087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb41780-ee69-4da0-96e8-fd8539e1bfe3 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:51.525024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1360/2709] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:52.432635 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:52.515089 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b807d8-1b68-4714-9d67-1183ad00f583 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:52.520527 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:52.521744 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:52.522232 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:52.522768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] PATH_INFO: `/v3/users/d109f2d02ab4459991da612505922461` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:52.524666 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d109f2d02ab4459991da612505922461)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:52.528899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:52.600808 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd109f2d02ab4459991da612505922461'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:52.613047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd109f2d02ab4459991da612505922461'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:52.618289 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:52.629296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd109f2d02ab4459991da612505922461'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:52.654712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd109f2d02ab4459991da612505922461'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:52.655863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd109f2d02ab4459991da612505922461'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:52.667546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1350/2710] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:52 2022] DELETE /identity/v3/users/d109f2d02ab4459991da612505922461 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:52.674409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:52.675330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:52.675330 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:52.712524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:52.714177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:52.790497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:52.814562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1361/2711] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:52.828145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:52.829927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:52.920475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e40-6f59-4c2d-a92f-f88c4038b4ea admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:52.927267 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:52.928485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:52.929039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:52.929638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:52.943036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:52.945234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:52.960713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1351/2712] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:53.595345 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:53.596918 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:53.610797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec570e-ad97-4ce6-9cfa-37801bc32b33 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:53.616731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:53.617867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:53.618048 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:53.618332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] PATH_INFO: `/v3/projects/b987cd8f98aa490db88a846caeee7212` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:53.619682 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b987cd8f98aa490db88a846caeee7212)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:53.622044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:53.706644 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b987cd8f98aa490db88a846caeee7212', 'request_id': 'req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.718782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b987cd8f98aa490db88a846caeee7212', 'request_id': 'req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.731650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b987cd8f98aa490db88a846caeee7212', 'request_id': 'req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.752968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] The token cache is being invalidate because project b987cd8f98aa490db88a846caeee7212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:53.753635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.754877 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.757630 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1362/2713] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:53 2022] DELETE /identity/v3/projects/b987cd8f98aa490db88a846caeee7212 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:31:53.763889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:53.777604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2d6d2-1605-4f47-9591-88776a0b94be service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:53.870019 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:53.870622 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:53.870905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:53.871178 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] PATH_INFO: `/v3/users/4267aa21b3f44cb98469638b56c87659` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:53.872819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4267aa21b3f44cb98469638b56c87659)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:53.874853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:53.935376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4267aa21b3f44cb98469638b56c87659'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.946759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4267aa21b3f44cb98469638b56c87659'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.957973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4267aa21b3f44cb98469638b56c87659'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.981639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4267aa21b3f44cb98469638b56c87659'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.982499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bd403482-972d-4c28-a7e5-1d7ed2cdc3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4267aa21b3f44cb98469638b56c87659'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:53.984708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:53.996785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1352/2714] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:31:53 2022] DELETE /identity/v3/users/4267aa21b3f44cb98469638b56c87659 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:54.528482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:54.530961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:54.556288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a0ad-1a86-447c-9eb9-4b37ecfcb80d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:54.659818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:54.660858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:54.661245 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:54.661560 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] PATH_INFO: `/v3/projects/dff63980e5294b57a803118aea20d68e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:54.662968 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff63980e5294b57a803118aea20d68e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:54.664978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:54.746924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff63980e5294b57a803118aea20d68e', 'request_id': 'req-b6a3c5e0-44ca-447c-9700-bde4672b7634'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:54.758908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff63980e5294b57a803118aea20d68e', 'request_id': 'req-b6a3c5e0-44ca-447c-9700-bde4672b7634'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:54.772289 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff63980e5294b57a803118aea20d68e', 'request_id': 'req-b6a3c5e0-44ca-447c-9700-bde4672b7634'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:54.790206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] The token cache is being invalidate because project dff63980e5294b57a803118aea20d68e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:31:54.790791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:54.791537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b6a3c5e0-44ca-447c-9700-bde4672b7634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:31:54.793124 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1363/2715] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:31:54 2022] DELETE /identity/v3/projects/dff63980e5294b57a803118aea20d68e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:54.799268 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:54.800004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:54.800004 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:54.823212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:54.824975 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:54.901970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1353/2716] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:31:54.911515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:54.911845 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:54.912167 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:54.928886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:54.936876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:54.939953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:54.954847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1364/2717] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:31:54.959853 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:54.971296 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a52552-7d92-4939-b6b0-6e327c4bbf26 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:54.976282 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:54.976973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:54.977253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:54.977566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:54.978317 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:31:54.978624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:54.980360 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:54.995439 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1354/2718] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:31:55.002692 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.019830 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed90b06-f3db-413d-a63c-4f55a98b9847 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.028234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.029525 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.030029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.030518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.031793 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.035680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.079676 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1365/2719] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:31:55 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:31:55.085981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.100803 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b12c-abdd-4e1a-9bb6-4507bb5b9a24 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.193028 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.193898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.194120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.194404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.195260 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.197421 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.224770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1355/2720] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:31:55 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:31:55.229286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.242748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01769-1a39-4202-b424-39b3daf470db admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.248618 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.249549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.249941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.250337 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.251234 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.253165 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.266131 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1366/2721] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:55.271447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.289229 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c24b-9bbd-4e8d-b596-aa3ed0cb9572 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.299366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.299366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.299867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.300036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.301524 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.304612 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09124214-dec9-4f0f-8c4d-f2f8c54b2281 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.324998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1356/2722] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:31:55.330075 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.342164 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706778e6-0eec-417f-bd56-c678bcf56e95 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.349810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.351695 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.352097 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.352557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] PATH_INFO: `/v3/projects/74d32d3ad70b4a94b3aefb57689a82e7/users/17a867e060c543c1bff959d17ffc637d/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.372753 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d32d3ad70b4a94b3aefb57689a82e7, user_id=17a867e060c543c1bff959d17ffc637d, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.375973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.392635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1367/2723] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:31:55 2022] PUT /identity/v3/projects/74d32d3ad70b4a94b3aefb57689a82e7/users/17a867e060c543c1bff959d17ffc637d/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:31:55.467859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47f70-f124-49d7-88de-444629b7ee45 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.468146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47f70-f124-49d7-88de-444629b7ee45 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.468403 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae47f70-f124-49d7-88de-444629b7ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.507192 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6ae47f70-f124-49d7-88de-444629b7ee45 None None] MFA Rules not processed for user `17a867e060c543c1bff959d17ffc637d`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:55.508602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6ae47f70-f124-49d7-88de-444629b7ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:55.582840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:55.604581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1357/2724] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:31:55.617492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:55.619486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:55.689037 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49433-3bb1-4f5d-9114-e62bf137688e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:55.693723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:55.694626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:55.694952 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:55.695321 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:55.706529 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:55.708189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:55.719948 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1368/2725] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:31:56.775052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf919e6-fce4-4ab3-9237-b2f336a7bb9a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:56.776849 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf919e6-fce4-4ab3-9237-b2f336a7bb9a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:56.777246 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf919e6-fce4-4ab3-9237-b2f336a7bb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:56.818607 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3bf919e6-fce4-4ab3-9237-b2f336a7bb9a None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:56.820006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3bf919e6-fce4-4ab3-9237-b2f336a7bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:56.914149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1358/2726] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:56.993470 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:56.995903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:57.131374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5e926-ce3c-4659-b3e1-75f8c63a6e14 service glance] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:57.138661 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:57.139421 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.139713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.139987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.153446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:57.155565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6bf6a-bdd6-4957-8660-4a783ebadc2f service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:57.168110 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1369/2727] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:31:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:31:57.248400 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.249288 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.249753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.251494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1359/2728] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:31:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:31:57.262987 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719db8e-442f-482d-acf2-01f69b32a759 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.263782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719db8e-442f-482d-acf2-01f69b32a759 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.264413 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719db8e-442f-482d-acf2-01f69b32a759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.305957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b719db8e-442f-482d-acf2-01f69b32a759 None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:57.308310 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b719db8e-442f-482d-acf2-01f69b32a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:57.396992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:31:57.463942 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1370/2729] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:31:57.502145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:57.505945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:57.578194 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad10f5-e554-40a5-8af1-fcb409c640c0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:57.597980 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:57.599191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.599696 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.600211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.628926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:57.631367 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbe2fa-fab7-48ec-a82f-ab52eb8a018c service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:57.644565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1360/2730] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:31:57.795637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc202f4-4659-4f69-a516-6102ab4b622f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.796465 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc202f4-4659-4f69-a516-6102ab4b622f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.797246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc202f4-4659-4f69-a516-6102ab4b622f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.800658 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1371/2731] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:31:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:31:57.809800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da805e-581b-429b-a766-ff2035f5f787 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:57.810376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da805e-581b-429b-a766-ff2035f5f787 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:57.810790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da805e-581b-429b-a766-ff2035f5f787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:57.843475 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e4da805e-581b-429b-a766-ff2035f5f787 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:31:57.845137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e4da805e-581b-429b-a766-ff2035f5f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:31:57.998662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1361/2732] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:31:58.000704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc202f4-4659-4f69-a516-6102ab4b622f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:31:58.034867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc202f4-4659-4f69-a516-6102ab4b622f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:31:58.055839 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:31:58.057083 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:31:58.058326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:31:58.059017 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:31:58.074118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:31:58.079069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d6bce-eaa1-415a-97f7-ab0dd9a37d93 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:31:58.099371 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1372/2733] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:32:07.956409 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=69 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:08.087462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=13 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:14.793209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:14.793209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:14.793209 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:14.795013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1362/2734] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:32:14.802460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c5f9d-5129-4b82-9e3f-a5c7b8c5eec9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:14.803561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c5f9d-5129-4b82-9e3f-a5c7b8c5eec9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:14.803561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c5f9d-5129-4b82-9e3f-a5c7b8c5eec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:14.845835 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-919c5f9d-5129-4b82-9e3f-a5c7b8c5eec9 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:14.847670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-919c5f9d-5129-4b82-9e3f-a5c7b8c5eec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:14.890946 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1373/2735] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:32:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:14.906102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:14.908147 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:14.924995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafce6c-ce4a-45d3-9957-adc0acb6ea12 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:14.929222 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:14.929998 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:14.930285 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:14.930573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:14.943370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:14.944898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3f24-0686-4c7a-a21e-1e016271bb1e service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:14.958390 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1363/2736] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:32:15.055296 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.055756 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.056251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.058043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1374/2737] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:32:15.065382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1eeb-071b-422a-9e80-aa5fcda2a66a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.066056 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1eeb-071b-422a-9e80-aa5fcda2a66a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.066627 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d1eeb-071b-422a-9e80-aa5fcda2a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.111953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-059d1eeb-071b-422a-9e80-aa5fcda2a66a None None] MFA Rules not processed for user `8ff3caa305504fcd888458b91bf1fc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:15.114018 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-059d1eeb-071b-422a-9e80-aa5fcda2a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:15.127182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1364/2738] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:32:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:15.148108 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:15.151591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:15.167214 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e395a-2f53-4551-9f70-e5c98542322b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:15.172192 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:15.175883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.175883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.175883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.183875 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:15.185817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643794f6-5627-41fa-ba26-6b580b1887ef service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:15.202089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1375/2739] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:32:15.338228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4790-46d0-42da-9527-134a3b1848af None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.338842 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4790-46d0-42da-9527-134a3b1848af None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.339655 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f4790-46d0-42da-9527-134a3b1848af None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.343105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1365/2740] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:32:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:32:15.353125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c0ee-fc38-42b0-a047-4edcd8819e44 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.353870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c0ee-fc38-42b0-a047-4edcd8819e44 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.354382 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49c0ee-fc38-42b0-a047-4edcd8819e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.409545 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9f49c0ee-fc38-42b0-a047-4edcd8819e44 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:15.412447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f49c0ee-fc38-42b0-a047-4edcd8819e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:15.435629 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1376/2741] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:32:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:32:15.458266 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f4790-46d0-42da-9527-134a3b1848af None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:15.472191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f4790-46d0-42da-9527-134a3b1848af None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:15.481724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:15.482539 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:15.482761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:15.483045 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:15.513751 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:15.516759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:15.537907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1366/2742] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:32:20.384723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:20.385705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:20.385937 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:20.412236 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:20.413903 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:20.424853 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1377/2743] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:20.444015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:20.465121 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177cf5a-71df-4d0d-961a-8cc9bd43b7bd service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:20.473693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:20.474764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:20.475181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:20.475580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:20.486210 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:20.488355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86630bac-3529-41ca-8740-619ddc391782 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:20.498778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1367/2744] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:32:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:32:23.782396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:23.906781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00c3f-2285-4ef3-97dd-926a1716b89f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:23.915542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:23.917200 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:23.917759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:23.918278 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] PATH_INFO: `/v3/users/17a867e060c543c1bff959d17ffc637d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:23.920748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a867e060c543c1bff959d17ffc637d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:23.923066 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:24.017633 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a867e060c543c1bff959d17ffc637d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:24.033652 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a867e060c543c1bff959d17ffc637d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:24.034807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a867e060c543c1bff959d17ffc637d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:24.050068 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a867e060c543c1bff959d17ffc637d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:24.050868 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a867e060c543c1bff959d17ffc637d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:24.125898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1378/2745] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:32:23 2022] DELETE /identity/v3/users/17a867e060c543c1bff959d17ffc637d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:32:24.127801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:24.128668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:24.128668 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:24.153926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:24.155699 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:24.338565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1368/2746] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:24.354905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:24.357536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:24.456282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:24.559389 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f686c-7f3a-414b-9326-065c0ea746c0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:24.565847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:24.567014 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:24.567562 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:24.568249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:24.590543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:24.593288 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acd5cb-3460-4aa4-ae5a-a8aa8229fbf5 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:24.614123 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1379/2747] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:32:25.202511 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:25.206326 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:25.243779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744a85f-13fd-4508-ad46-f1aa502b1890 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:25.271397 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:25.272905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:25.273431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:25.273905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] PATH_INFO: `/v3/projects/74d32d3ad70b4a94b3aefb57689a82e7` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:25.276543 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d32d3ad70b4a94b3aefb57689a82e7)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:25.280793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:25.388047 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d32d3ad70b4a94b3aefb57689a82e7', 'request_id': 'req-2dc3fb83-ac95-4733-bba8-80512e712cfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:25.405005 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d32d3ad70b4a94b3aefb57689a82e7', 'request_id': 'req-2dc3fb83-ac95-4733-bba8-80512e712cfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:25.436278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d32d3ad70b4a94b3aefb57689a82e7', 'request_id': 'req-2dc3fb83-ac95-4733-bba8-80512e712cfb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:25.489015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] The token cache is being invalidate because project 74d32d3ad70b4a94b3aefb57689a82e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:32:25.490706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:25.492311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2dc3fb83-ac95-4733-bba8-80512e712cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:25.502746 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:25.503795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:25.504363 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:25.513457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1369/2748] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:32:25 2022] DELETE /identity/v3/projects/74d32d3ad70b4a94b3aefb57689a82e7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:32:25.550243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:25.551794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:25.720152 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1380/2749] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:25.729417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:25.729879 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:25.730381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:25.776557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:25.777948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:25.788118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1370/2750] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:25.793999 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:25.826730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e6e9b-74f3-4fa9-a197-4250d7dcc0b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:25.835025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:25.836148 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:25.836613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:25.837065 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:25.838446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:32:25.839141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:25.843348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:25.875183 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1381/2751] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:32:25.880060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:25.892012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a65a-fba6-46ae-8505-671625ff1ebe None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:25.897867 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:25.898532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:25.898814 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:25.899432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:25.907532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:25.912815 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:25.981576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1371/2752] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:32:25 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:32:25.984452 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:26.008724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980abbd-691a-4cef-8259-1b168cc7ea9f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:26.121035 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:26.122252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:26.122730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:26.123193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:26.124595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:26.128759 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:26.171825 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1382/2753] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:32:25 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:32:26.176819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:26.198783 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d622d-b925-4d75-b3a1-f1d921806b40 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:26.207541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:26.208741 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:26.209498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:26.210154 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:26.211707 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:26.215113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:26.219787 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:26.243491 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1372/2754] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:32:26.247549 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:26.267012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bc8dc-1712-4d33-880c-532d1644a0e9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:26.272651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:26.273510 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:26.273900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:26.274429 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:26.275159 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:26.277446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f8bb-8bb6-4bb6-b005-9a70b4aff924 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:26.308287 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1383/2755] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:32:26.313546 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:26.333755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2b08-2f7d-42d9-9443-44521a10f532 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:26.342486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:26.344569 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:26.345211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:26.345851 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] PATH_INFO: `/v3/projects/a5f55b7fb858497ca39585b915dd2173/users/7a59985f5c9a4ea382bbc64fd609a9cc/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:26.536813 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f55b7fb858497ca39585b915dd2173, user_id=7a59985f5c9a4ea382bbc64fd609a9cc, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:26.539925 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:26.570396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1373/2756] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:32:26 2022] PUT /identity/v3/projects/a5f55b7fb858497ca39585b915dd2173/users/7a59985f5c9a4ea382bbc64fd609a9cc/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:32:26.582613 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:26.584320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:26.585258 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:26.636636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] MFA Rules not processed for user `7a59985f5c9a4ea382bbc64fd609a9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:26.639995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:26.675365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:26.824043 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1384/2757] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:26.854650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:26.858138 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:27.018826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b371c-31bb-4320-bfbe-71f4692e3410 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:27.025169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:27.026465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:27.027022 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:27.028444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:27.044493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:27.047727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b61b-64f0-44a2-9aae-d319c3b5e67b service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:27.067665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1374/2758] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:32:27.156253 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:27.312722 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e4b95-fc00-47f3-bb03-5e4fefd638d1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:27.318415 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:27.319374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:27.319650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:27.319920 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] PATH_INFO: `/v3/users/7a59985f5c9a4ea382bbc64fd609a9cc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:27.321298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a59985f5c9a4ea382bbc64fd609a9cc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:27.323651 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:27.441811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a59985f5c9a4ea382bbc64fd609a9cc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:27.462936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a59985f5c9a4ea382bbc64fd609a9cc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:27.464189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a59985f5c9a4ea382bbc64fd609a9cc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:27.483300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a59985f5c9a4ea382bbc64fd609a9cc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:27.484029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a59985f5c9a4ea382bbc64fd609a9cc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:27.540966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1385/2759] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:32:27 2022] DELETE /identity/v3/users/7a59985f5c9a4ea382bbc64fd609a9cc => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:32:27.550477 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:27.551218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:27.551759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:27.627245 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:27.629982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:27.744608 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1375/2760] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:27.757965 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:27.759973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:27.849802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:27.902030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde5af0-f970-42b0-aea9-59a7ef6fa289 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:27.906713 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:27.907424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:27.907748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:27.907995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:27.920125 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:27.921733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54877e-f1cd-4d45-95de-da923a2c9d6e service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:27.937453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1386/2761] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:32:28.530650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:28.533456 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:28.552638 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbe2f7-7bc7-4c63-8fe1-d029f4dc751d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:28.559537 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:28.563090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:28.563090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:28.563090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] PATH_INFO: `/v3/projects/a5f55b7fb858497ca39585b915dd2173` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:28.564988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f55b7fb858497ca39585b915dd2173)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:28.567770 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:28.634523 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f55b7fb858497ca39585b915dd2173', 'request_id': 'req-40224698-583e-4808-8d79-eecbd749b2d3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:28.646191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f55b7fb858497ca39585b915dd2173', 'request_id': 'req-40224698-583e-4808-8d79-eecbd749b2d3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:28.653294 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f55b7fb858497ca39585b915dd2173', 'request_id': 'req-40224698-583e-4808-8d79-eecbd749b2d3'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:28.662819 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:28.670621 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] The token cache is being invalidate because project a5f55b7fb858497ca39585b915dd2173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:32:28.671079 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:28.672198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:32:28.674488 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1376/2762] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:32:28 2022] DELETE /identity/v3/projects/a5f55b7fb858497ca39585b915dd2173 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:32:37.929158 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=9 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:40.316941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:40.317861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:40.317861 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:40.341067 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:32:40.342790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:40.413881 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1387/2763] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:32:40.428532 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:32:40.430885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:32:40.519414 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40224698-583e-4808-8d79-eecbd749b2d3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:32:40.525723 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:32:40.526422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:32:40.526698 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:32:40.526972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:32:40.538200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:32:40.539774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80640bb0-a61f-4bb1-8daa-de1be5d39b64 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:32:40.550172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1377/2764] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:32:50.405084 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=25 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:32:50.545796 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=32 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:18.114446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:18.116796 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:18.273004 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290df40-8e5d-4286-a3fe-e787cf84fe54 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:18.281329 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:18.282468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:18.282931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:18.283397 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:18.286665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:18.308340 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:18.311500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:18.332852 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1388/2765] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:33:23.811863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef70e4-3fa8-42dd-9a65-1e1f499c4107 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:23.812497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef70e4-3fa8-42dd-9a65-1e1f499c4107 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:23.812497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef70e4-3fa8-42dd-9a65-1e1f499c4107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:23.835157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d7ef70e4-3fa8-42dd-9a65-1e1f499c4107 None None] MFA Rules not processed for user `37e7fecdaae94842b6917152fb7f1598`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:33:23.836722 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d7ef70e4-3fa8-42dd-9a65-1e1f499c4107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:23.909977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1378/2766] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:33:24.365040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:24.367515 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:24.436391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21bed-3a18-47e7-b9c6-70e5c4fcaeb3 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:24.443812 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:24.444866 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:24.445292 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:24.445794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:24.448878 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:24.466836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:24.469312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:24.484993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1389/2767] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:33:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:33:32.525128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:32.525827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:32.525979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:32.546882 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] MFA Rules not processed for user `d3eebc82ce654013998f58585c123a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:33:32.548133 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:32.627981 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1379/2768] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:33:34.477779 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=67 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:42.621283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=50 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:44.459670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:44.470979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04ce93-8154-4e8a-bbae-650ea3c444de service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:44.477464 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:44.478675 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:44.479174 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:44.479667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:44.483184 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:44.591563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:44.593286 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:44.608854 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1390/2769] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:33:54.603792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=30 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:56.247385 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:56.252083 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:56.314812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd5a4d-6f9b-4e01-b88b-ef199cc3edc0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:56.340706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:56.344342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:56.344342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:56.344342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] PATH_INFO: `/v3/users/6a52a93774cd48868f9b5656821b0f12` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:56.346656 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a52a93774cd48868f9b5656821b0f12)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:56.351118 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:56.514583 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a52a93774cd48868f9b5656821b0f12'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:56.532872 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a52a93774cd48868f9b5656821b0f12'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:56.556516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a52a93774cd48868f9b5656821b0f12'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:56.598179 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a52a93774cd48868f9b5656821b0f12'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:56.599963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a52a93774cd48868f9b5656821b0f12'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:56.615322 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1380/2770] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:33:56 2022] DELETE /identity/v3/users/6a52a93774cd48868f9b5656821b0f12 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:33:57.044982 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:57.066403 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e44494-861e-41c7-be33-5fa738c88312 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:57.201955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:57.203347 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:57.203694 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:57.204189 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] PATH_INFO: `/v3/projects/90317adf2c86461d96439dc48441b6c8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:57.206578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90317adf2c86461d96439dc48441b6c8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:57.210262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:57.316479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90317adf2c86461d96439dc48441b6c8', 'request_id': 'req-0ede8b86-3051-41d8-94c5-66031ee7b57d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.332571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90317adf2c86461d96439dc48441b6c8', 'request_id': 'req-0ede8b86-3051-41d8-94c5-66031ee7b57d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.352171 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90317adf2c86461d96439dc48441b6c8', 'request_id': 'req-0ede8b86-3051-41d8-94c5-66031ee7b57d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.379879 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] The token cache is being invalidate because project 90317adf2c86461d96439dc48441b6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:33:57.380474 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.381375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.383810 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1391/2771] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:33:57 2022] DELETE /identity/v3/projects/90317adf2c86461d96439dc48441b6c8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:33:57.389503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:57.391885 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:57.413898 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea665688-2306-4e62-ba4f-04aca4790f60 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:57.549907 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:57.551460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:57.552010 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:57.552528 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] PATH_INFO: `/v3/users/069913534b5c4d5c9c96acae8f24f84b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:57.555160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069913534b5c4d5c9c96acae8f24f84b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:57.559639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:57.691038 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069913534b5c4d5c9c96acae8f24f84b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.707531 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069913534b5c4d5c9c96acae8f24f84b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.724586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069913534b5c4d5c9c96acae8f24f84b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.744168 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:57.758320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069913534b5c4d5c9c96acae8f24f84b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.761953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069913534b5c4d5c9c96acae8f24f84b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:57.795120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1381/2772] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:33:57 2022] DELETE /identity/v3/users/069913534b5c4d5c9c96acae8f24f84b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:33:58.188971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:33:58.195147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:33:58.234059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede8b86-3051-41d8-94c5-66031ee7b57d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:33:58.373956 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:33:58.375575 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:33:58.375821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:33:58.376173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] PATH_INFO: `/v3/projects/ad3e87f106944a95990be3ec4e4da21f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:33:58.385858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad3e87f106944a95990be3ec4e4da21f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:33:58.389928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:33:58.463551 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:33:58.514536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad3e87f106944a95990be3ec4e4da21f', 'request_id': 'req-5158db30-044c-4048-8c0b-388b79e48bd6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:58.529282 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad3e87f106944a95990be3ec4e4da21f', 'request_id': 'req-5158db30-044c-4048-8c0b-388b79e48bd6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:58.546791 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad3e87f106944a95990be3ec4e4da21f', 'request_id': 'req-5158db30-044c-4048-8c0b-388b79e48bd6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:58.575847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] The token cache is being invalidate because project ad3e87f106944a95990be3ec4e4da21f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:33:58.576430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:58.577815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:33:58.580940 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1392/2773] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:33:58 2022] DELETE /identity/v3/projects/ad3e87f106944a95990be3ec4e4da21f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:34:07.789377 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:34:08.567402 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:34:23.759381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:23.761053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:23.842899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec5cd-12f1-417e-b327-b4f014eb5fc1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:23.906431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:23.907354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:23.907562 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:23.907866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:23.910141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:23.998396 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:23.999868 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d482a-ad1b-4da7-8621-8d3a23bb2f1b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:24.013283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1382/2774] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:34:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:34:27.718100 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:27.720012 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:27.734576 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158db30-044c-4048-8c0b-388b79e48bd6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:27.739597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:27.740359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:27.740778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:27.741229 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:27.743784 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:27.759928 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:27.761723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:27.775704 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1393/2775] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:34:33.866879 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47525f-71c9-43e2-b968-36d2910cbac3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:33.867516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47525f-71c9-43e2-b968-36d2910cbac3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:33.867516 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47525f-71c9-43e2-b968-36d2910cbac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:33.896163 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-da47525f-71c9-43e2-b968-36d2910cbac3 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:34:33.897637 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da47525f-71c9-43e2-b968-36d2910cbac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:33.973560 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1383/2776] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:34:34.040225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:34.042917 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:34.176479 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23fb7-6475-448d-b5c1-a30458b9243d service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:34.186118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:34.186995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:34.187482 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:34.187978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:34.204649 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:34.207794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168befeb-7a6e-49d4-9d79-465e35680148 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:34.227591 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1394/2777] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:34:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:34:43.967176 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:34:44.219744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=38 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:34:49.324821 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:49.325790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:49.325790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:49.328195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1384/2778] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 25 16:34:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 16:34:49.336291 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:49.337509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:49.337900 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:49.402512 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] MFA Rules not processed for user `d73b8801206a4973a97fda3fb00f55c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:34:49.404552 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:49.421020 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1395/2779] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 25 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:34:49.438305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:49.440021 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:49.538054 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab7779-0620-491e-99c2-99162d2c1ffd None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:49.544321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:49.545521 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:49.545976 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:49.546460 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:49.565703 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:49.568418 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:49.589200 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1385/2780] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:34:49.820089 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:49.822503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:49.914565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a2a69-288a-4041-9b27-538743696346 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:49.920309 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:49.921116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:49.921486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:49.921817 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:49.923995 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:49.937790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:49.939938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec0dfe-98f7-488c-b500-10350deea57b service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:49.952168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1396/2781] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:34:50.761031 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:50.763737 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:50.777617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c706544-9f21-4106-af96-869d9f0351f9 service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:50.782947 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:50.783801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:50.784197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:50.784577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:50.787128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:50.799328 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:50.800978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:50.811590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1386/2782] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:34:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:34:51.696597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0832-e362-4893-a0eb-42db962eb4db None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:51.697320 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0832-e362-4893-a0eb-42db962eb4db None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:51.697524 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0832-e362-4893-a0eb-42db962eb4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:51.763938 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2c4b0832-e362-4893-a0eb-42db962eb4db None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:34:51.765366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2c4b0832-e362-4893-a0eb-42db962eb4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:34:51.787079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1397/2783] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 25 16:34:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:34:51.804386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:34:51.835800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21876-7c5e-4301-aa1f-ef62f327abdf service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:34:51.844792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:34:51.845753 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:34:51.846250 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:34:51.846728 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:34:51.863774 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:34:51.867197 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:34:51.885422 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1387/2784] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:34:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:35:01.781555 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=59 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:01.878120 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=42 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:06.268862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6de8-bbbc-420d-b67a-dbb30a886749 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:06.269751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6de8-bbbc-420d-b67a-dbb30a886749 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:06.269751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f6de8-bbbc-420d-b67a-dbb30a886749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:06.295637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b35f6de8-bbbc-420d-b67a-dbb30a886749 None None] MFA Rules not processed for user `6b4413d1949d4f88935b5f3c39c0ac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:35:06.296886 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b35f6de8-bbbc-420d-b67a-dbb30a886749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:06.305518 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1398/2785] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 25 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:35:06.319495 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:06.331207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a583-8874-464a-93eb-200484887f3a service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:06.336647 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f36fd1f37394b49ac38568200578c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:06.337304 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:06.337616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:06.337894 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:06.347261 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:06.348997 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:06.359315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1388/2786] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 25 16:35:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 16:35:16.289706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:16.354590 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:37.098137 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:37.099252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:37.099252 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:37.120805 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] MFA Rules not processed for user `d3eebc82ce654013998f58585c123a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:35:37.122365 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:37.192760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1399/2787] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:35:39.349844 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:39.352379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:39.442327 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaacf9-8dc3-414e-bc79-07a3dabff77b service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:39.447339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:39.448091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:39.448455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:39.448863 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:39.451043 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:39.528550 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:39.530040 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:39.541141 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1389/2788] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:42.713767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:42.716044 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:42.785527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d5b51-8668-4fac-90f9-26f17270097b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:42.796843 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:42.798634 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:42.799266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:42.799858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:42.808142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:42.862934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:42.867685 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:42.921623 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1400/2789] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:47.924301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:47.945095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b67b7-1da5-4293-b555-4684646ca19a service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:47.951880 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:47.953035 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:47.953573 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:47.954060 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:47.957513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:48.093611 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:48.095659 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:48.111221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1390/2790] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:52.855180 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:52.877224 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170728cc-f92e-4413-af3e-6e20b74df17c service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:52.886542 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:52.887706 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:52.888204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:52.888705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:52.892047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:52.909966 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:52.912314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:52.926430 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1401/2791] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:52.942680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:52.946937 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:52.962688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cfa17-252b-49ac-921c-cb82a9b75d49 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:52.971899 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:52.973624 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:52.974180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:52.974714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] PATH_INFO: `/v3/users/37e7fecdaae94842b6917152fb7f1598` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:52.977085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e7fecdaae94842b6917152fb7f1598)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:52.980457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:53.049644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:53.054756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e7fecdaae94842b6917152fb7f1598'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.067632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e7fecdaae94842b6917152fb7f1598'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.079265 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e7fecdaae94842b6917152fb7f1598'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.098095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e7fecdaae94842b6917152fb7f1598'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.098977 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e7fecdaae94842b6917152fb7f1598'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.107586 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1391/2792] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:35:52 2022] DELETE /identity/v3/users/37e7fecdaae94842b6917152fb7f1598 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:35:53.158751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:53.181391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf7b58-b651-4018-9e4e-3570acace7a5 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:53.277653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:53.278811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.279304 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.279787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.282978 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:53.376673 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:53.379784 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:53.403375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:53.406668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1082a-4f1c-4aad-8460-92abbcd0bccf service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:53.422313 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1402/2793] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:53.509315 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17344111-322f-444a-8c6a-98972872d345 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:53.516044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:53.517675 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.518159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.518616 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] PATH_INFO: `/v3/projects/4501489e56744cf98e49895b6d5aa6f8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.520609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4501489e56744cf98e49895b6d5aa6f8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:53.522809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:53.591576 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4501489e56744cf98e49895b6d5aa6f8', 'request_id': 'req-18f9fdef-69e4-4d0b-8da3-7e03c177d231'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.601840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4501489e56744cf98e49895b6d5aa6f8', 'request_id': 'req-18f9fdef-69e4-4d0b-8da3-7e03c177d231'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.610478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4501489e56744cf98e49895b6d5aa6f8', 'request_id': 'req-18f9fdef-69e4-4d0b-8da3-7e03c177d231'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.624362 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] The token cache is being invalidate because project 4501489e56744cf98e49895b6d5aa6f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:35:53.624639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.625548 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18f9fdef-69e4-4d0b-8da3-7e03c177d231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:35:53.627429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1392/2794] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:35:53 2022] DELETE /identity/v3/projects/4501489e56744cf98e49895b6d5aa6f8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:35:53.635350 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.635783 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.636090 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.659626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:35:53.661008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:53.674561 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:53.755776 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1403/2795] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:35:53.764901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.765230 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.765559 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.787786 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:35:53.789172 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:53.799591 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1393/2796] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:35:53.804537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:53.821398 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24280d38-ab34-45db-9be2-f85c97bc0480 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:53.830536 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:53.831571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.831859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.832051 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.832707 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:35:53.833040 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:53.835303 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:53.849919 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1404/2797] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:35:53.854940 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:53.866337 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2eeee-11ff-4fa7-932d-da03124bcf83 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:53.872461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:53.873086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:53.873405 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:53.873669 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:53.874478 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:53.876329 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:53.910416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1394/2798] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:35:53.916824 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:53.928353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b635b-be2f-4fc5-89d4-8b9c526bcd41 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:54.002665 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:54.004112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.004112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.004112 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.004889 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:54.007015 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:54.033734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1405/2799] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:35:53 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:35:54.039837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:54.054299 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec9b73-03c1-471a-a9bd-3230c890fc63 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:54.059883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:54.060757 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.061132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.061464 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.062416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:54.064497 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:54.077436 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1395/2800] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:35:54.082034 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:54.093668 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a29aa-b05d-4b3a-a619-9f357c665c2f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:54.099227 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:54.099964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.100249 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.100526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.101271 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:54.103120 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ab949-4731-40e4-94d8-32a2ce6c834f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:54.118785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1406/2801] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:35:54.124856 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:54.137100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5007f8d-38dd-46fe-9598-97fd94ea27bd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:54.142445 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:54.143376 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.143708 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.143990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] PATH_INFO: `/v3/projects/7ad628e26f9349e59d73f6606242d957/users/1f86305f296440bcae5268e1a0c8b089/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.154724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:54.166449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad628e26f9349e59d73f6606242d957, user_id=1f86305f296440bcae5268e1a0c8b089, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:54.170248 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:54.192855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1396/2802] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:35:54 2022] PUT /identity/v3/projects/7ad628e26f9349e59d73f6606242d957/users/1f86305f296440bcae5268e1a0c8b089/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:35:54.199205 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.199565 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.199887 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.222475 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:35:54.223811 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:54.301632 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1407/2803] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:35:54.314763 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:54.316983 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:54.396423 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54d71-3400-4aa4-a7f2-dcf3fcd3c34f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:54.401662 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:54.402443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:54.402792 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:54.403114 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:54.416251 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:54.417897 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c908116-7f55-44c0-9d74-dc6be8742b7a service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:54.430102 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1397/2804] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:35:57.166193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:35:57.168924 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:57.224838 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:35:57.332582 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556d732-6bfa-4228-9992-be47452a14a8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:35:57.344944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:35:57.346507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:35:57.347360 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:35:57.347566 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:35:57.351377 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:35:57.370460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:35:57.373609 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:35:57.393908 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1408/2805] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:35:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:36:00.468809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4f0c-bf77-47a7-901c-4a9d0507f39e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:36:00.470252 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4f0c-bf77-47a7-901c-4a9d0507f39e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:36:00.470978 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4f0c-bf77-47a7-901c-4a9d0507f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:36:00.524825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d19e4f0c-bf77-47a7-901c-4a9d0507f39e None None] MFA Rules not processed for user `1f86305f296440bcae5268e1a0c8b089`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:36:00.527419 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d19e4f0c-bf77-47a7-901c-4a9d0507f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:36:00.672551 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1398/2806] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:36:00.699008 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:36:00.701212 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:36:00.720469 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71f6d-d989-47da-abf2-f092e98fe1b1 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:36:00.727641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:36:00.728395 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:36:00.728674 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:36:00.728953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:36:00.739662 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:36:00.741412 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:36:00.756640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1409/2807] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:36:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:36:03.036214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:36:03.037846 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:36:03.038613 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:36:03.078066 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] MFA Rules not processed for user `1f86305f296440bcae5268e1a0c8b089`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:36:03.081142 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:36:03.100203 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1399/2808] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:36:03.766738 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:36:03.780962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfca7-34ff-451c-ba4d-38319199e619 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:36:03.785563 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:36:03.786648 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:36:03.787098 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:36:03.787533 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:36:03.792126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:36:03.809901 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:36:03.811559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b64817-297a-455e-9541-43430ebaf2ed service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:36:03.823577 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1410/2809] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:36:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:36:04.134404 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:36:04.159932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c586459-d8cc-435b-b63c-50ac3296660d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:36:04.170561 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:36:04.172274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:36:04.174049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:36:04.174848 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:36:04.197961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:36:04.219218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:36:04.220944 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:36:04.245534 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1400/2810] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:36:13.819622 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=36 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:36:14.229928 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:37:04.062265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:04.063204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:04.063204 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:04.096362 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] MFA Rules not processed for user `1f86305f296440bcae5268e1a0c8b089`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:37:04.098726 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:04.111493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1411/2811] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:37:14.087332 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:37:45.105727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:45.107547 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:45.121159 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9708b9-7681-493a-9061-c4df453e2fbe service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:45.125100 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:45.126211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:45.126718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:45.127149 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:45.129211 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:45.144759 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:45.147824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:45.167765 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1401/2812] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:37:53.624197 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:53.627568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:53.749636 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859b6b7-802f-49b5-a09c-9889da389bc8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:53.756372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:53.757381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:53.757787 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:53.758160 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:53.760392 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:53.780079 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:53.780388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:53.782281 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:53.784234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:53.797355 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1412/2813] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:37:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:37:53.846091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607edc63-36fb-4bfc-81f4-6e0308262558 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:53.869388 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430007-1872-47ce-8939-322290fe837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:53.873039 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430007-1872-47ce-8939-322290fe837f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:53.873639 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430007-1872-47ce-8939-322290fe837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:53.876577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430007-1872-47ce-8939-322290fe837f admin admin] PATH_INFO: `/v3/users/d3eebc82ce654013998f58585c123a0e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:53.884816 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37430007-1872-47ce-8939-322290fe837f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3eebc82ce654013998f58585c123a0e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:53.890366 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37430007-1872-47ce-8939-322290fe837f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:54.008199 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3eebc82ce654013998f58585c123a0e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.018565 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3eebc82ce654013998f58585c123a0e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.032371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3eebc82ce654013998f58585c123a0e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.066417 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3eebc82ce654013998f58585c123a0e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.068411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3eebc82ce654013998f58585c123a0e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.077465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1402/2814] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:37:53 2022] DELETE /identity/v3/users/d3eebc82ce654013998f58585c123a0e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:37:54.406141 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:54.431983 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eee25-e5c3-4077-b509-7348fb576c7c service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:54.522744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:54.524016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:54.524453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:54.524921 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] PATH_INFO: `/v3/projects/ee2c762eec4646afbd0ea4d71a58894d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:54.526553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee2c762eec4646afbd0ea4d71a58894d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:54.529906 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:54.602190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee2c762eec4646afbd0ea4d71a58894d', 'request_id': 'req-eedef0f1-d570-498d-83cf-d3c5b815aace'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.609626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee2c762eec4646afbd0ea4d71a58894d', 'request_id': 'req-eedef0f1-d570-498d-83cf-d3c5b815aace'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.617497 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee2c762eec4646afbd0ea4d71a58894d', 'request_id': 'req-eedef0f1-d570-498d-83cf-d3c5b815aace'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.634246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] The token cache is being invalidate because project ee2c762eec4646afbd0ea4d71a58894d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:37:54.635145 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.635416 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.637650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1413/2815] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:37:54 2022] DELETE /identity/v3/projects/ee2c762eec4646afbd0ea4d71a58894d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:37:54.642466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:54.644437 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:54.666726 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430007-1872-47ce-8939-322290fe837f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:54.759180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:54.760354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:54.760743 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:54.761124 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] PATH_INFO: `/v3/users/15200f16c51e42d59351b04a311797df` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:54.762694 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=15200f16c51e42d59351b04a311797df)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:54.766160 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:54.832001 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15200f16c51e42d59351b04a311797df'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.839382 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15200f16c51e42d59351b04a311797df'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.844030 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:37:54.851351 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15200f16c51e42d59351b04a311797df'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.886431 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15200f16c51e42d59351b04a311797df'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.887311 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e1055559-1869-436f-a2fe-54150d75e05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15200f16c51e42d59351b04a311797df'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:54.901411 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1403/2816] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:37:54 2022] DELETE /identity/v3/users/15200f16c51e42d59351b04a311797df => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:37:55.302290 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:37:55.305029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:37:55.328381 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef0f1-d570-498d-83cf-d3c5b815aace admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:37:55.358029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:37:55.427923 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:37:55.429747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:37:55.429747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:37:55.429747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] PATH_INFO: `/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:37:55.431319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5a80ea9f1b43d88e4fe96b8b87716e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:37:55.433446 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:37:55.509511 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5a80ea9f1b43d88e4fe96b8b87716e', 'request_id': 'req-856706ed-8104-4242-a7fe-83714da3f2e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:55.519488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5a80ea9f1b43d88e4fe96b8b87716e', 'request_id': 'req-856706ed-8104-4242-a7fe-83714da3f2e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:55.528556 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5a80ea9f1b43d88e4fe96b8b87716e', 'request_id': 'req-856706ed-8104-4242-a7fe-83714da3f2e7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:55.552700 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] The token cache is being invalidate because project da5a80ea9f1b43d88e4fe96b8b87716e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:37:55.553188 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:55.554353 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:37:55.557326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1414/2817] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:37:55 2022] DELETE /identity/v3/projects/da5a80ea9f1b43d88e4fe96b8b87716e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:38:04.890789 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:38:05.242041 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589e7a-8489-4000-8692-a2e08e9c07a1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:38:05.242498 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589e7a-8489-4000-8692-a2e08e9c07a1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:38:05.242857 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589e7a-8489-4000-8692-a2e08e9c07a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:38:05.267051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ba589e7a-8489-4000-8692-a2e08e9c07a1 None None] MFA Rules not processed for user `1f86305f296440bcae5268e1a0c8b089`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:38:05.268739 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ba589e7a-8489-4000-8692-a2e08e9c07a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:38:05.341469 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1404/2818] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:38:05.544451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:38:15.334190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=25 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:38:21.309139 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:38:21.310957 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:38:21.406751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856706ed-8104-4242-a7fe-83714da3f2e7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:38:21.415372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:38:21.416277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:38:21.416578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:38:21.416931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:38:21.419022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:38:21.434104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:38:21.436009 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:38:21.447717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1415/2819] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:38:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:38:31.443491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=32 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:39:09.843841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:39:09.844755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:39:09.844755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:39:09.868012 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] MFA Rules not processed for user `1f86305f296440bcae5268e1a0c8b089`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:39:09.869287 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:39:09.880927 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1405/2820] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:39:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:39:19.861302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:39:59.638126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:39:59.639800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:39:59.737846 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb187e-aadb-41f1-b4ee-0130f95ef671 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:39:59.744462 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:39:59.745641 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:39:59.746080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:39:59.746568 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:39:59.750064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:39:59.879933 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:39:59.881614 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:39:59.895220 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1416/2821] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:40:07.344506 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:07.364518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52e93-8613-4f89-83fc-15ceab222ec1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:07.371687 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:07.372861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:07.373387 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:07.373900 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:07.377771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:07.403837 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:07.406657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:07.430113 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1406/2822] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:09.889128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=54 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:17.417769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:23.184720 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:23.187177 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:23.421424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe2581-775f-4ee7-910c-89fd30521256 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:23.430507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:23.432118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:23.432621 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:23.433128 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] PATH_INFO: `/v3/users/1f86305f296440bcae5268e1a0c8b089` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:23.436473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f86305f296440bcae5268e1a0c8b089)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:23.440595 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:23.524016 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f86305f296440bcae5268e1a0c8b089'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:23.534635 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f86305f296440bcae5268e1a0c8b089'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:23.535667 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f86305f296440bcae5268e1a0c8b089'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:23.550277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f86305f296440bcae5268e1a0c8b089'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:23.551254 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2bfb2c97-66fe-47be-b3df-fd7d784f5939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f86305f296440bcae5268e1a0c8b089'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:23.583727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1417/2823] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:40:23 2022] DELETE /identity/v3/users/1f86305f296440bcae5268e1a0c8b089 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:23.816302 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:23.891355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb378d-8ec3-4697-b2a7-64d6c255693d service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:23.996061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:23.996061 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:23.996594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:23.996775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] PATH_INFO: `/v3/projects/7ad628e26f9349e59d73f6606242d957` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:23.998364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad628e26f9349e59d73f6606242d957)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.001036 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.082286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad628e26f9349e59d73f6606242d957', 'request_id': 'req-43ea703c-56d3-4db5-9c7c-2909f260c24f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:24.095274 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad628e26f9349e59d73f6606242d957', 'request_id': 'req-43ea703c-56d3-4db5-9c7c-2909f260c24f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:24.105000 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad628e26f9349e59d73f6606242d957', 'request_id': 'req-43ea703c-56d3-4db5-9c7c-2909f260c24f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:24.123742 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] The token cache is being invalidate because project 7ad628e26f9349e59d73f6606242d957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:40:24.124286 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:24.125432 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43ea703c-56d3-4db5-9c7c-2909f260c24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:24.128224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1407/2824] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:40:23 2022] DELETE /identity/v3/projects/7ad628e26f9349e59d73f6606242d957 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:24.134461 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.134828 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.135168 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.170979 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:24.222847 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:24.224198 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:24.301323 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1418/2825] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:40:24.311150 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.311712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.311712 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.334949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:24.336438 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:24.346381 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1408/2826] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:40:24.352007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.369955 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413c37d-f07b-4d3f-a4c5-9d9bf963d4cf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.378391 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.379603 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.380077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.380571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.382011 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:40:24.382605 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.385816 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.410215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1419/2827] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:40:24.415961 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.431610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a20fcf-ba54-4f07-84d1-ff982ab6af0d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.439793 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.440529 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.440800 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.441076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.441921 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.443918 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.507896 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1409/2828] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:40:24 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:24.514782 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.526971 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c06d-5839-4ee1-996f-7891429e97ad admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.600990 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.601973 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.602460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.602888 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.604103 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.608190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.640799 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1420/2829] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:40:24 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:24.644630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.658195 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9f419-7464-4c78-8420-535a29267c23 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.664764 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.665661 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.666084 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.666371 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.667335 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.669859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.694809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1410/2830] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:24.698072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.716226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7d67-1ef0-4ac2-ba7a-373fbae7853f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.724447 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.725531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.725962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.726384 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.727604 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.730451 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d3b37-c36c-4ec7-b481-8c50fa9811e6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.749543 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1421/2831] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:24.756911 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:24.771290 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdd06c-798c-4ad5-8456-dcd48c261e9d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:24.778901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:24.780709 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.781292 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.781889 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] PATH_INFO: `/v3/projects/4f2b5f69683944aab36e1ab4e00dc05a/users/8adffff6990840f0b2e4e611a762394d/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.812095 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2b5f69683944aab36e1ab4e00dc05a, user_id=8adffff6990840f0b2e4e611a762394d, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:24.817724 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:24.822341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:24.841750 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1411/2832] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:24 2022] PUT /identity/v3/projects/4f2b5f69683944aab36e1ab4e00dc05a/users/8adffff6990840f0b2e4e611a762394d/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:24.860025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:24.860504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:24.861042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:24.901130 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] MFA Rules not processed for user `8adffff6990840f0b2e4e611a762394d`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:24.902727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:24.962272 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:25.002262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1422/2833] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:25.015969 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:25.018448 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:25.153465 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e994243-a674-49f3-a649-fd7437fe8c38 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:25.162357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73b8801206a4973a97fda3fb00f55c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:25.163557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:25.163963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:25.164444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:25.179224 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:25.182354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aab98-6715-4c3d-8127-d7f018d34fdf service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:25.202942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1412/2834] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:25.483207 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:25.586553 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee25d42-f6c3-4030-9824-f1e8951cbf28 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:25.593905 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:25.595457 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:25.596092 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:25.596473 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] PATH_INFO: `/v3/users/8adffff6990840f0b2e4e611a762394d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:25.600959 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8adffff6990840f0b2e4e611a762394d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:25.605086 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:25.695581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8adffff6990840f0b2e4e611a762394d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:25.706276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8adffff6990840f0b2e4e611a762394d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:25.707492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8adffff6990840f0b2e4e611a762394d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:25.721731 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8adffff6990840f0b2e4e611a762394d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:25.722954 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8adffff6990840f0b2e4e611a762394d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:25.752154 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1423/2835] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:40:25 2022] DELETE /identity/v3/users/8adffff6990840f0b2e4e611a762394d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:25.758979 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:25.759341 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:25.759664 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:25.784824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:25.786425 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:25.870785 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1413/2836] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:25.884073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:25.886944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:25.985526 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:26.008760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547f73a-dc7c-44b7-91f3-4649afc32de0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:26.013964 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:26.014750 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.015142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.015494 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.026751 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:26.028567 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd74419-0541-416a-8550-f4e1887c1066 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:26.041523 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1424/2837] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 16:40:26.545127 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:26.548235 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:26.571754 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e395a9-846d-4e9f-83bd-7ee3e7906878 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:26.581128 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:26.582730 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.583190 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.583677 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] PATH_INFO: `/v3/projects/4f2b5f69683944aab36e1ab4e00dc05a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.586249 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f2b5f69683944aab36e1ab4e00dc05a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:26.590238 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:26.673320 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f2b5f69683944aab36e1ab4e00dc05a', 'request_id': 'req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:26.691766 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f2b5f69683944aab36e1ab4e00dc05a', 'request_id': 'req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:26.696305 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:26.702594 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f2b5f69683944aab36e1ab4e00dc05a', 'request_id': 'req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:26.726343 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] The token cache is being invalidate because project 4f2b5f69683944aab36e1ab4e00dc05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:40:26.726840 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:26.728003 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-77ac2481-d25c-4a1d-9cc6-cdc04ecae379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:26.731277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1414/2838] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:40:26 2022] DELETE /identity/v3/projects/4f2b5f69683944aab36e1ab4e00dc05a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:26.741686 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.742300 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.742781 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.772752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:26.774316 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:26.860113 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1425/2839] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:26.869771 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.870253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.870617 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.895342 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:26.896693 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:26.906177 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1415/2840] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:26.911481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:26.929802 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c6737-192c-430d-ba4d-ebbcfff4e04c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:26.937961 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:26.939099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.939610 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.940088 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.941519 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:40:26.942061 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:26.945265 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:26.969077 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1426/2841] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:40:26.974214 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:26.986430 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8200f-efd4-42fd-b77a-bcb4d3a1a017 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:26.994015 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:26.994779 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:26.995044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:26.995347 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:26.996108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:26.998408 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.032974 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1416/2842] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:40:26 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:27.040049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.058859 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe47555-de15-4028-b152-663f3bee6b7f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.172909 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.174022 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.174319 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.174626 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.175496 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.178030 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.207612 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1427/2843] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:40:27 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:27.216631 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.235307 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6923b7-7b75-42fa-acb2-039efbec869b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.243904 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.245145 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.245706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.246228 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.247769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.251275 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.269447 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1417/2844] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:27.275099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.295298 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97db7a-a7bc-4580-9035-2d5d047fd509 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.302334 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.303226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.303723 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.304118 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.305453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.307795 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbe397-6397-4d9e-8568-b36f998a4718 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.324508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1428/2845] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:27.332384 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.347941 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bbd85-1969-43f8-a412-243a79125aa0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.356485 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.358428 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.358903 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.359386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] PATH_INFO: `/v3/projects/eebb4c580679426d883463a7b5a970aa/users/154ebf928b4447639cd59e792860fffe/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.389221 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebb4c580679426d883463a7b5a970aa, user_id=154ebf928b4447639cd59e792860fffe, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.401864 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.429920 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1418/2846] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:27 2022] PUT /identity/v3/projects/eebb4c580679426d883463a7b5a970aa/users/154ebf928b4447639cd59e792860fffe/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:27.452420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.452911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.453437 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.509147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] MFA Rules not processed for user `154ebf928b4447639cd59e792860fffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:27.511481 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:27.589747 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:27.630671 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1429/2847] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:40:27.638212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.761002 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1fe-928e-462b-90b0-bf6c4430608e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.769852 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.770990 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.771500 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.772239 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.773541 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.777226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:27.826790 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1419/2848] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:40:27 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:27.832907 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:27.852797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647cc4-5312-437d-8ec8-0073caa37580 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:27.965513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:27.967073 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:27.967326 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:27.967840 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:27.969640 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:27.973557 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.013728 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1430/2849] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:40:27 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:28.019091 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.032181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d503d-565f-4af4-8fa1-690d96cf9331 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.039193 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.040108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.040494 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.040953 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.041948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.044046 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.064339 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1420/2850] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:28.069305 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.080670 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d52-3e08-4e8d-960b-215eac089a24 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.088509 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.090062 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.090537 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.091024 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] PATH_INFO: `/v3/projects/42f274f0871240138d10b2779b1461a2/users/2067ea53d01d456388cf42ac28c5a857/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.112528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f274f0871240138d10b2779b1461a2, user_id=2067ea53d01d456388cf42ac28c5a857, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.115571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.134914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1431/2851] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:28 2022] PUT /identity/v3/projects/42f274f0871240138d10b2779b1461a2/users/2067ea53d01d456388cf42ac28c5a857/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:40:28.141782 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.160357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5427658-7738-4945-86b1-186b6f0a1042 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.249841 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:28.263526 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.264769 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.265714 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.267014 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.268718 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.274074 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c79eed-4fe5-434e-9c10-17633b3201c2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.295861 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1421/2852] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:28.303264 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.323617 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121449ba-ad5d-4bd8-9997-6c863151de9a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.333989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.335507 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.335993 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.336488 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] PATH_INFO: `/v3/projects/42f274f0871240138d10b2779b1461a2/users/2067ea53d01d456388cf42ac28c5a857/roles/8b8b7516fb6f40bd878cd90fb40874ea` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.340374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f274f0871240138d10b2779b1461a2, user_id=2067ea53d01d456388cf42ac28c5a857, role_id=8b8b7516fb6f40bd878cd90fb40874ea)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.345934 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.362898 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1432/2853] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:28 2022] PUT /identity/v3/projects/42f274f0871240138d10b2779b1461a2/users/2067ea53d01d456388cf42ac28c5a857/roles/8b8b7516fb6f40bd878cd90fb40874ea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:28.382620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.383111 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.383604 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.431336 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] MFA Rules not processed for user `2067ea53d01d456388cf42ac28c5a857`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:28.433518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:28.554999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1422/2854] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:28.573911 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.577870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:28.701080 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed2ab-586a-4c0e-abcd-0594c4a5d20f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.708059 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015923-105d-45a8-910d-93de798745ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.709243 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015923-105d-45a8-910d-93de798745ce service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.709762 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015923-105d-45a8-910d-93de798745ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.710225 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015923-105d-45a8-910d-93de798745ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.727760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015923-105d-45a8-910d-93de798745ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.730559 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015923-105d-45a8-910d-93de798745ce service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.749992 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1433/2855] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:28.798277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:28.893644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07cbb2-7ab7-4099-83c8-10b3aab2fe6f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:28.899346 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:28.900416 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:28.900809 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:28.901185 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] PATH_INFO: `/v3/users/154ebf928b4447639cd59e792860fffe` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:28.902755 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=154ebf928b4447639cd59e792860fffe)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:28.905300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:28.978455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154ebf928b4447639cd59e792860fffe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:28.989103 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154ebf928b4447639cd59e792860fffe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.004093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154ebf928b4447639cd59e792860fffe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.032316 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154ebf928b4447639cd59e792860fffe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.033593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154ebf928b4447639cd59e792860fffe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.036234 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:29.048513 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1423/2856] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:40:28 2022] DELETE /identity/v3/users/154ebf928b4447639cd59e792860fffe => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:29.055579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:29.056460 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:29.056639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:29.085739 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:29.087894 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:29.143275 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:29.207314 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1434/2857] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:29.224355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:29.227033 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:29.340357 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cfb21-4063-47dc-a561-b331ef637bda admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:29.347226 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:29.348455 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:29.348932 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:29.349466 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:29.368772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:29.371901 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:29.392866 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1424/2858] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:29.820794 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:29.823294 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:29.854444 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2875-831f-4262-a33c-a80ac176c874 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:29.864597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:29.866199 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:29.866705 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:29.867142 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] PATH_INFO: `/v3/projects/eebb4c580679426d883463a7b5a970aa` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:29.869637 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebb4c580679426d883463a7b5a970aa)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:29.873580 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:29.969284 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebb4c580679426d883463a7b5a970aa', 'request_id': 'req-ad58d18b-73e9-440b-b911-332b94dab782'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.980745 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebb4c580679426d883463a7b5a970aa', 'request_id': 'req-ad58d18b-73e9-440b-b911-332b94dab782'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:29.994578 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebb4c580679426d883463a7b5a970aa', 'request_id': 'req-ad58d18b-73e9-440b-b911-332b94dab782'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.019052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] The token cache is being invalidate because project eebb4c580679426d883463a7b5a970aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:40:30.019546 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.020729 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.023531 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1435/2859] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:40:29 2022] DELETE /identity/v3/projects/eebb4c580679426d883463a7b5a970aa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:30.029824 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:30.047825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b5c6-b5f4-4b07-bf53-7edaabaf88b3 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:30.172006 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:30.173514 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:30.174137 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:30.174620 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] PATH_INFO: `/v3/users/2067ea53d01d456388cf42ac28c5a857` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:30.176988 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2067ea53d01d456388cf42ac28c5a857)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:30.180860 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:30.264429 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2067ea53d01d456388cf42ac28c5a857'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.274264 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2067ea53d01d456388cf42ac28c5a857'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.285555 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2067ea53d01d456388cf42ac28c5a857'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.301685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2067ea53d01d456388cf42ac28c5a857'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.302580 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7f58eac1-ff54-498a-a523-d28a8816622f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2067ea53d01d456388cf42ac28c5a857'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.310218 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1425/2860] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:40:30 2022] DELETE /identity/v3/users/2067ea53d01d456388cf42ac28c5a857 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:40:30.683072 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:30.686312 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:30.706232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58d18b-73e9-440b-b911-332b94dab782 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:30.781589 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:30.782493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:30.782778 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:30.783060 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] PATH_INFO: `/v3/projects/42f274f0871240138d10b2779b1461a2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:30.784608 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f274f0871240138d10b2779b1461a2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:30.786653 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:30.855841 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f274f0871240138d10b2779b1461a2', 'request_id': 'req-a2422428-676a-4e2c-8058-3b7379ad4466'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.864433 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f274f0871240138d10b2779b1461a2', 'request_id': 'req-a2422428-676a-4e2c-8058-3b7379ad4466'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.866855 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:30.873969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f274f0871240138d10b2779b1461a2', 'request_id': 'req-a2422428-676a-4e2c-8058-3b7379ad4466'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.894233 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] The token cache is being invalidate because project 42f274f0871240138d10b2779b1461a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:40:30.894639 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.895501 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2422428-676a-4e2c-8058-3b7379ad4466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:30.897366 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1436/2861] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:40:30 2022] DELETE /identity/v3/projects/42f274f0871240138d10b2779b1461a2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:30.903184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:30.903482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:30.903826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:30.928667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:30.929973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:30.946987 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:31.019595 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1426/2862] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:31.028276 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.028588 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.028821 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.053380 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:31.054715 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:31.063733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1437/2863] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:31.068680 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.080811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa02c-bf33-4796-affd-9151604a2936 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.087067 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.087811 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.088140 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.088575 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.089640 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:40:31.089970 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.091797 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.106545 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1427/2864] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:40:31.112950 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.130266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa5b37-3581-4244-b8bf-1bbf9d34d411 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.138581 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.139541 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.140025 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.140513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.141914 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.145807 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.189190 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1438/2865] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:40:31 2022] POST /identity/v3/projects => generated 359 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:31.195778 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.210301 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6939b-b5a9-4439-acc4-857ff72b8dc3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.284107 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.284829 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.285112 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.285424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.286146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.288184 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.316781 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1428/2866] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:40:31 2022] POST /identity/v3/users => generated 280 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 16:40:31.321953 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.335013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634752af-ff0c-47ef-9a3c-aebdc57d7ec7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.341367 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.342359 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.342680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.343064 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.343935 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.345883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.358836 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1439/2867] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:31.364053 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.381293 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32542d-c71a-4eae-bcdd-6f1bd0115dd1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.390181 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.391180 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.391602 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.392085 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.393394 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.396283 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f61f7-4a0f-4a19-bf51-bab962579fe5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.409833 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1429/2868] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:31.415283 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.427232 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee4e6-76d8-4e21-b10d-c0b58c60518a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.432453 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.433439 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.433832 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.434222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] PATH_INFO: `/v3/projects/a0ebb20ff8344138acefc4da04f58984/users/0647da6189d441cca10a138121fdfd3c/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.456471 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ebb20ff8344138acefc4da04f58984, user_id=0647da6189d441cca10a138121fdfd3c, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.462169 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.487170 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1440/2869] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:31 2022] PUT /identity/v3/projects/a0ebb20ff8344138acefc4da04f58984/users/0647da6189d441cca10a138121fdfd3c/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 16:40:31.499486 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.499826 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.500258 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.533888 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] MFA Rules not processed for user `0647da6189d441cca10a138121fdfd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:31.535493 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:31.615444 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1430/2870] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:31.658374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.661484 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:31.745492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3493bb-7f14-4dd8-bc33-2af35b3c5684 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.749680 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.750534 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.750858 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.751222 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.768050 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.771196 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d12b5-95ff-4ffc-bde2-bf77a3bbf712 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:31.788913 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1441/2871] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:31.811253 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:31.845300 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:31.932685 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657179-c274-4870-9f19-bc5fca32f4a1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:31.940935 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:31.942363 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:31.942812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:31.943278 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] PATH_INFO: `/v3/users/0647da6189d441cca10a138121fdfd3c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:31.945768 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0647da6189d441cca10a138121fdfd3c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:31.949705 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:32.032136 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0647da6189d441cca10a138121fdfd3c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.043052 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0647da6189d441cca10a138121fdfd3c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.057308 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0647da6189d441cca10a138121fdfd3c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.088231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0647da6189d441cca10a138121fdfd3c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.089482 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0647da6189d441cca10a138121fdfd3c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.100982 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1431/2872] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:40:31 2022] DELETE /identity/v3/users/0647da6189d441cca10a138121fdfd3c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:32.108078 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:32.108513 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:32.108893 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:32.135931 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:32.136246 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:32.240800 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1442/2873] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:40:32.254354 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:32.256115 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:32.345713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b7892-1c74-4abf-8910-33df79a4a0a8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:32.350051 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:32.350847 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:32.351157 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:32.351479 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:32.363721 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:32.365231 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a828149-7bd0-4f39-b34b-d8cfcc5cd594 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:32.377609 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1432/2874] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:32.835748 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:32.838193 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:32.863867 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ee9c-6245-45cb-aebd-d8135a13c820 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:32.873493 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:32.875500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:32.875797 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:32.876215 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] PATH_INFO: `/v3/projects/a0ebb20ff8344138acefc4da04f58984` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:32.878625 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ebb20ff8344138acefc4da04f58984)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:32.882558 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:32.923173 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:32.964047 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ebb20ff8344138acefc4da04f58984', 'request_id': 'req-164e2ab0-df47-4967-bdf4-e687b8202b44'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.971941 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ebb20ff8344138acefc4da04f58984', 'request_id': 'req-164e2ab0-df47-4967-bdf4-e687b8202b44'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:32.979870 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ebb20ff8344138acefc4da04f58984', 'request_id': 'req-164e2ab0-df47-4967-bdf4-e687b8202b44'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:33.001986 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] The token cache is being invalidate because project a0ebb20ff8344138acefc4da04f58984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:40:33.002500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:33.003579 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-164e2ab0-df47-4967-bdf4-e687b8202b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:40:33.006269 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1443/2875] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:40:32 2022] DELETE /identity/v3/projects/a0ebb20ff8344138acefc4da04f58984 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:33.014657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.015187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.015818 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.051713 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:33.053756 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:33.108619 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:33.152665 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1433/2876] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:33.164752 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.165262 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.165777 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.200277 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:33.202396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:33.217116 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1444/2877] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:40:33.229688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.242323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623446a-1302-4679-9fe5-b619ef7f87ea None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.249318 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.250303 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.250706 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.251090 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.251945 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:40:33.252353 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.254519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.270233 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1434/2878] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:40:33.277095 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.291774 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2c085-91da-4c47-912b-1f415b9d635c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.298069 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.298844 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.299126 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.299504 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.300528 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.302880 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.343717 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1445/2879] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:40:33 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:33.349182 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.360772 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772cc17-5af7-4166-b7bb-7f8548da7185 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.430942 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.431657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.431938 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.432364 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.432929 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.435013 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.459711 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1435/2880] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:40:33 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:40:33.464974 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.480042 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10ef7-560e-4699-a6d8-d2af4c98c778 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.486500 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.487335 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.487721 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.488085 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.488962 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.491241 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.504673 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1446/2881] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 16:40:33.511457 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.527934 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74177338-8a9a-4a50-a51f-7804ddfd33c4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.536076 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.537164 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.537634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.538049 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.539198 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.541949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283b2a1-e970-4172-a6b8-a054816d7abe admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.562731 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1436/2882] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:40:33.569091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.580374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe45968-faaf-4e66-9969-d224216d8c3a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.586520 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.587466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.587744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.588013 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] PATH_INFO: `/v3/projects/ba02bebcc0c34524b4d68f40b9e6b7ca/users/1016d9c1eb47409f84a9d94353e31e67/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.605876 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba02bebcc0c34524b4d68f40b9e6b7ca, user_id=1016d9c1eb47409f84a9d94353e31e67, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.608969 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.625597 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1447/2883] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:40:33 2022] PUT /identity/v3/projects/ba02bebcc0c34524b4d68f40b9e6b7ca/users/1016d9c1eb47409f84a9d94353e31e67/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:40:33.636566 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.636972 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.637254 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.671270 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] MFA Rules not processed for user `1016d9c1eb47409f84a9d94353e31e67`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:40:33.672727 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:33.747667 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1437/2884] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:40:33.761424 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:40:33.763147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:40:33.859550 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9770f1-0047-4a80-be7e-16ff3e26751a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:40:33.863710 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c159db46d48429b95e0166596c53d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:40:33.864492 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:40:33.865091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:40:33.865091 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:40:33.879348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:40:33.881883 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5e7ac-4972-4589-b616-cb0675d6050c service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:40:33.892689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1448/2885] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:40:43.744379 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:40:43.887882 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=77,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:05.133483 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:05.224330 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e99e3-a696-4c64-be62-b809c3c83802 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:05.232536 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:05.234086 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:05.234563 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:05.235075 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] PATH_INFO: `/v3/users/1016d9c1eb47409f84a9d94353e31e67` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:05.237093 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1016d9c1eb47409f84a9d94353e31e67)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:05.240277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:05.306225 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1016d9c1eb47409f84a9d94353e31e67'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:05.315459 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1016d9c1eb47409f84a9d94353e31e67'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:05.325812 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1016d9c1eb47409f84a9d94353e31e67'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:05.356877 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1016d9c1eb47409f84a9d94353e31e67'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:05.358461 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1016d9c1eb47409f84a9d94353e31e67'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:05.369443 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1438/2886] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:41:05 2022] DELETE /identity/v3/users/1016d9c1eb47409f84a9d94353e31e67 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:41:05.377033 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:05.377689 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:05.378049 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:05.411790 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:41:05.413944 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:05.524374 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1449/2887] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:41:05.535044 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:05.536651 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:05.605886 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299240c7-0889-44ef-a04f-7e3b9c94f16f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:05.610834 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:05.611577 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:05.611859 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:05.612139 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:05.623355 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:05.624761 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a740f-ffd8-4a77-aa0e-b0f8c96d6953 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:05.636169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1439/2888] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:41:05.981730 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:05.983508 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:06.000785 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b36114-92fb-410e-a373-fd6c92bcf64e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.007007 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.008104 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.008574 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.008989 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] PATH_INFO: `/v3/projects/ba02bebcc0c34524b4d68f40b9e6b7ca` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.010862 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba02bebcc0c34524b4d68f40b9e6b7ca)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.013029 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.080947 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba02bebcc0c34524b4d68f40b9e6b7ca', 'request_id': 'req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:06.088019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba02bebcc0c34524b4d68f40b9e6b7ca', 'request_id': 'req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:06.095733 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba02bebcc0c34524b4d68f40b9e6b7ca', 'request_id': 'req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:06.115396 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] The token cache is being invalidate because project ba02bebcc0c34524b4d68f40b9e6b7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:41:06.115775 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:06.116503 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-74a7df91-cb4d-4ba3-8d03-0ef485deeb49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:06.118280 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1450/2889] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:41:05 2022] DELETE /identity/v3/projects/ba02bebcc0c34524b4d68f40b9e6b7ca => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:41:06.124169 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.124557 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.124883 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.153481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:41:06.155481 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:06.187926 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:06.252098 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1440/2890] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:41:06.261693 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.262019 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.262358 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.286003 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:41:06.287527 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:06.302540 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1451/2891] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 16:41:06.307515 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.318905 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6ac49-d28a-403a-910e-fbbbcec79336 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.324212 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.325007 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.325277 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.325801 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.326499 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 16:41:06.326962 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.328950 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.343108 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1441/2892] 10.222.0.66 () {58 vars in 1153 bytes} [Mon Jul 25 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 16:41:06.349266 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.360378 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a23d3-077e-4c98-9197-3cbd4a74807b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.365724 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.366405 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.366773 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.367052 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.367772 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.369976 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.403734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1452/2893] 10.222.0.66 () {60 vars in 1153 bytes} [Mon Jul 25 16:41:06 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:41:06.408632 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.419948 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d4fec-09c6-4235-bdd0-fa938eb41fcb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.515963 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.517132 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.517674 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.518187 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.519441 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.523191 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.553775 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1442/2894] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 25 16:41:06 2022] POST /identity/v3/users => generated 282 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 16:41:06.558849 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.571744 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cd2ef-cb67-44e1-9cd5-061e340f37c7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.577463 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.578206 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.578491 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.578767 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.579514 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.581208 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.593741 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1453/2895] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:41:06.598365 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.610011 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f760-6b9e-440d-a855-06cb3f43a09f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.615105 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.615825 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.616094 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.616378 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.617119 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.618827 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449fe28-fa95-489c-8c31-e78a0bb64987 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.633134 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1443/2896] 10.222.0.66 () {58 vars in 1122 bytes} [Mon Jul 25 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 16:41:06.638792 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.651268 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdf8cf-030e-4811-ab7a-53b22d5baed5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.657099 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.658087 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.658372 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.658650 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] PATH_INFO: `/v3/projects/f68b482f90dd4789989cea528074bd2d/users/6919f53ee2e641eab750d5f9658dba1c/roles/c618a32897ba4eb08e80389e7b3d1cc9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.677318 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68b482f90dd4789989cea528074bd2d, user_id=6919f53ee2e641eab750d5f9658dba1c, role_id=c618a32897ba4eb08e80389e7b3d1cc9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.680147 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.697242 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1454/2897] 10.222.0.66 () {60 vars in 1483 bytes} [Mon Jul 25 16:41:06 2022] PUT /identity/v3/projects/f68b482f90dd4789989cea528074bd2d/users/6919f53ee2e641eab750d5f9658dba1c/roles/c618a32897ba4eb08e80389e7b3d1cc9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:41:06.710370 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.710689 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.710973 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.745236 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] MFA Rules not processed for user `6919f53ee2e641eab750d5f9658dba1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:41:06.746519 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:06.827269 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1444/2898] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:41:06.844587 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:06.846727 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:06.872770 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:06.919419 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf871a55-763d-4c77-a962-cd23675eccdb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:06.923548 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468fc86f361a4c7c91cacc1d90086176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:06.924251 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:06.925226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:06.925226 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:06.935420 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:06.937213 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92fbdb-97db-4cb9-aa7a-62566f24de15 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:06.948571 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1455/2899] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:41:16.823503 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:16.943348 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=17 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:41.528325 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:41.602087 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba5d45-197c-4e41-93cd-1ea9891c1107 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:41.607735 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:41.608650 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:41.608922 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:41.609207 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] PATH_INFO: `/v3/users/6919f53ee2e641eab750d5f9658dba1c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:41.610644 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6919f53ee2e641eab750d5f9658dba1c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:41.612646 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:41.678581 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6919f53ee2e641eab750d5f9658dba1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:41.687688 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6919f53ee2e641eab750d5f9658dba1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:41.698914 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6919f53ee2e641eab750d5f9658dba1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:41.718823 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6919f53ee2e641eab750d5f9658dba1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:41.719593 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6919f53ee2e641eab750d5f9658dba1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:41.726719 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1445/2900] 10.222.0.66 () {58 vars in 1224 bytes} [Mon Jul 25 16:41:41 2022] DELETE /identity/v3/users/6919f53ee2e641eab750d5f9658dba1c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:41:41.731657 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:41.732936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:41.732936 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:41.754407 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] MFA Rules not processed for user `3c939fd3ba4449cf949d6e5bd73effec`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 16:41:41.755598 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:41.831244 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1456/2901] 10.222.0.66 () {58 vars in 958 bytes} [Mon Jul 25 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 16:41:41.842855 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:41.844657 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:41.916995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43abd0f-32f0-4ff8-abfe-369a91e2456b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:41.921174 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff3caa305504fcd888458b91bf1fc5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:41.921995 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:41.922323 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:41.922610 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:41.933321 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:41.934949 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:41.946272 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1446/2902] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:41:42.348818 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:41:42.350487 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:41:42.364827 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71500cf-10bb-4a99-825f-e94b620f1b27 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:41:42.371486 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c939fd3ba4449cf949d6e5bd73effec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8ee5c3834cf43d4a8d8dd7aee49ae24', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:41:42.372837 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:41:42.373375 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:41:42.373815 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] PATH_INFO: `/v3/projects/f68b482f90dd4789989cea528074bd2d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:41:42.375409 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68b482f90dd4789989cea528074bd2d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:41:42.378449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:41:42.437734 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68b482f90dd4789989cea528074bd2d', 'request_id': 'req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:42.447468 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68b482f90dd4789989cea528074bd2d', 'request_id': 'req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:42.455627 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68b482f90dd4789989cea528074bd2d', 'request_id': 'req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:42.477466 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] The token cache is being invalidate because project f68b482f90dd4789989cea528074bd2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 16:41:42.477760 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:42.478449 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5a2c209c-fbfe-459c-9bb7-6a07252fec0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 16:41:42.480467 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1457/2903] 10.222.0.66 () {58 vars in 1233 bytes} [Mon Jul 25 16:41:42 2022] DELETE /identity/v3/projects/f68b482f90dd4789989cea528074bd2d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 16:41:51.941424 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:41:52.472896 dv-cinder-836973-15-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 16:43:25.358146 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 16:43:25.360386 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:43:25.489912 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde73f-5fe6-4a65-9159-d1b2e7b7a250 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 16:43:25.564634 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf1c53cb8784f5c9a2d8d2d71751fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0729a317dec4f6a9639c0492b388eba', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 16:43:25.565449 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 16:43:25.565791 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 16:43:25.566186 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 16:43:25.568520 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 16:43:25.582630 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 16:43:25.584999 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964dc83-00e3-4607-a7df-82c5a710ae46 service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 16:43:25.598518 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1447/2904] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 25 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 16:43:35.592589 dv-cinder-836973-15-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=67 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}